Programmer.ie: Modern AI programming

A site about AI programming

Home
AI
Finance
Programming
Lists
About

Policy Networks

Models From First Principles 03: SICQL — Building a Model From Q, V and Policy Networks

Recent Posts

Advanced Agents From First Principles 09: Can Your Agent Actually Learn From Previous Runs?
Advanced Agents From First Principles 06: Does One Agent Plan, Execute and Judge Its Own Work? Build a Planner-Executor-Critic Architecture
Advanced Agents From First Principles 05: Is One Model Doing Everything? Build a Mixture of Experts at the Agent Level
Advanced Agents From First Principles 04: Does Your Agent Prune Good Ideas Too Early? Use Monte Carlo Tree Search for Long-Horizon Reasoning
Advanced Agents From First Principles 03: Does Your Agent Commit to a Bad Reasoning Path Too Early? Build a Tree of Thoughts
Advanced Agents From First Principles 02: Why Does My Reasoning Agent Give a Different Answer Every Time? Use Self-Consistency Without Confusing Consensus With Truth
Advanced Agents From First Principles 01: Does Your AI Agent Fail on Complex Reasoning Tasks? Treat Chain of Thought as Computation, Not Proof
Advanced Agents From First Principles 00: When Should You Use an Advanced Agent Architecture?
Agents From First Principles 09: AI Agent Says It Worked When It Didn’t? Verify the Result Outside the LLM
Agents From First Principles 08: AI Agent Picks the First Solution? Add Search Instead of One-Shot Generation
Agents From First Principles 07: AI Agent Forgets Previous Work? Add Working, Semantic and Episodic Memory
Agents From First Principles 06: AI Agent Chooses the Wrong Tool? Design Better Tool Interfaces, Schemas and Routing
Agents From First Principles 05: AI Agent Gets Stuck in a Loop? Add State, Feedback and Stopping Conditions
Agents From First Principles 04: AI Agent Fails on Multi-Step Tasks? Separate Planning From Execution
Agents From First Principles 03: AI Agent Keeps Making the Same Mistake? Add a Critique-and-Revision Loop
Agents From First Principles 02: AI Agent Gives Inconsistent Answers? Generate Multiple Candidates and Rank Them
AI Agent Returning Invalid Tool Calls? How to Validate LLM Actions
Agents From First Principles 00: What Is an Agent, Really?
Models From First Principles 08: Which Model Should You Use? MR.Q, EBT, SICQL, HRM, Tiny and PACS Compared
Models From First Principles 07: PACS — Building an Optimizer From Gradient Statistics
Models From First Principles 06: Inside Tiny — Residual Blocks, Attention and Sparse Autoencoders
Models From First Principles 05: Tiny — Recursive Reasoning With a Small Neural Network
Models From First Principles 04: HRM — Hierarchical Reasoning With Fast and Slow Recurrent State
Models From First Principles 03: SICQL — Building a Model From Q, V and Policy Networks
Models From First Principles 02: EBT — From One Score to Q, V, Policy and Advantage

Categories

Tags

Ablation Study Absorbing Knowledge ACL-Verbatim Adaptive Agents Adaptive AI Adaptive AI Pipelines Adaptive AI Systems Adaptive Reasoning Adaptive Reasoning Depth Adaptive Runtime Systems Advantage Advantage-Weighted Learning Advantage-Weighted Regression Adversarial Evaluation Adversarial Review Adversarial Validation Agent Agent Architecture Agent Collaboration Agent Debugging Agent Evaluation Agent Loops Agent Memory Agent Orchestration Agent Planning Agent Routing Agent Search Agent State Agent Swarms Agent-Based Training Loops Agentic AI Agentic Systems Agentic Tree Search Agentic Tree Search Agents AGI AI AI Adoption AI Agents AI Amplification AI and Society AI Application Gap AI Architecture AI Art AI as an Amplifier AI Audit Layer AI Backlash AI Capex AI Co-Scientist AI Coding AI Compression AI Debugging AI Detection AI Distillation AI Distrust AI Economics AI Economy AI Engineering AI Evolution AI Future AI Hallucination AI Image Critique AI Infrastructure AI Memory Systems AI Model Evolution AI Observability AI Orchestration Ai Paper Walkthrough AI Pipelines AI Productivity AI Prototyping AI Provenance AI Reasoning AI Reasoning Framework AI Reasoning Systems AI Reliability AI Research Systems AI Safety AI Scalability AI Self-Evaluation AI Self-Improvement AI Singularity AI System Design AI Systems Design AI Training AI vs Human AI Writing AI Writing Systems AI-Assisted Health AI-Assisted Research Alignment via Embeddings Answering Machine Effect Anthropic Append-Only Logs Arena Competition ARM Artificial Intelligence Arxiv Asset Inflation Asset Prices Asset-Holder Enrichment Asset-Price State Associative Memory Assumption Testing Asymmetric Incidence Attention Attribution Attribution Hallucination Auditable Reasoning Auditable Search Authenticity Crisis Autogptq Autograd Automated Research Agents Autonomous Intelligence Autonomous Research Agents Autonomous Retraining Autoround Backpropagation Beam Search Belief Cartridges Best-of-N Betti Numbers Big Tech Bitsandbytes Blossom Engine Bond Yields Bond-Market Confidence Broadcasting Buffers Calibration Canada Economy Canada Housing Candidate Ranking Capability Systems Capability vs Deployment Capacity Constraints Capacity Growth Capital Gains Capital Inflows Capital per Worker Cargo Cult Cognition Cartridges Case-Based Reasoning Causal Modeling CBR Chain of Rubrics Chain of Thought Chronic Illness Citation Verification CiteVQA Citizen Balance Sheet Citizen Solvency Citizen Solvency Model Civilizational Systems Claim Verification Claim-Driven Search Claude Code CLIP CMHC Cnn Co-Ai Framework Code Generation Code Search CodeBERT Codex Coding Agents Cognitive Amplification Cognitive Architecture Cognitive Architectures Cognitive Automation Cognitive Garden Cognitive Graph Cognitive Heartbeat Cognitive Patterns Cognitive Planning Cognitive Runtime Cognitive Search Cognitive State Engineering Cognitive Surgery Command Line Common Sense Reasoning Computer Vision Confidence Tracking Consistency Score Constitutional Reasoning Construction Throughput Content Explosion Content-Addressable Cognition Context Engineering Context Sensitivity Context-Filtered Reasoning Contradiction Detector Contrastive Learning Controlled Reasoning Conv2d Coq Corporate Tax Ireland Corporation Tax Concentration Cosine Similarity Cost of Living CoT CoT Encyclopedia Cot vs Debate Critic Agent Critical Thinking Critique Cross-Domain Reasoning Transfer Cross-Modal Analysis CSO Data Cuda Curriculum Learning Cycle Detection Dashboard Dashboard Growth Dashboard State Dataloader Debt Cycle Debt Legitimacy Debt Rollover Debt Sustainability Debt-Service Burden Debugging Decentralization Decision Frameworks Decision Lineage Deep Learning Deep Learning Deployment Deep Learning Limitations Delta Field Delta Memory Dependency vs Independence Deterministic Gating Deterministic Replay Developer Productivity Developer Tools Digital Biomarkers Dimension-Wise Evaluation Dimensional Scoring Distributed AI Systems Document Embedding Strategies Document Profiling Document Scoring Dollar System Domain Classification Domestic Productive Capacity DPO DSPy Dual-Pass Scoring Dynamic Causal Model Dynamic Chunking Dynamic Model Loading Dynamic Reasoning Dynamic Scoring EBT Scoring Economic Constraint Economic Measurement Economic Signals EDGAR Edge AI Editing Electricity Economy Embedding Alignment Embedding Comparison Embedding Compatibility Embedding Protocol Design Embedding Subspaces Embedding-Based Reasoning Embedding-Based Training Embedding-Based Tuning Embeddings Energy-Based Models Enterprise AI Entity Recognition Entrant Exclusion Episodic Memory Epistemic Architecture Epistemic Engines Epistemic Fields Epistemic Memory Epistemic Quality Epistemic Scaffolding Epistemic State Eurozone Constraint Evaluation Evaluation Frameworks Event Sourcing Event-Sourced Cognition Evidence Engine Evidence Memory Evidence Quicksand Evidence-Based AI Evidence-Based Reasoning Evidence-Bound Synthesis Evidential Reasoning Executable Cognition Executable Cognitive Kernel Executable Search Execution Traces Experience Compiler Explainable AI Explainable Analysis External Constraint Extractive QA Extractive Transfer Extrinsic Hallucination Failure Taxonomy False Acceptance Rate Falsification Family Formation FastAPI Feature Extraction Feature Importance Federal Revenue Federal-Provincial Dynamics Feedback Loops Feedback Loops in AI FFmpeg Filmstrip Fin-R1 Finance Financial Repression Fine-Tuning Fiscal Constraint Fiscal Policy Forecasting Formal Verification Format Selection Fp16 Fuel Protests Function Calling Functional Intelligence Functions Future Citizen Balance Sheet Future of Work GAP Component GDP per Capita General Reasoner Geometric Machine Learning GILD Integration GILD Training GNI vs GDP Goal-Based Learning Goal-Conditioned Evaluation Goal-Conditioned Scoring Goal-Driven AI Goodhart’s Law Gpt Gpu Gradients Graph of Thoughts Graph-Based Memory Graph-Based Reasoning GrepSeek Grounded Verification GRPO Gru GSM8K H-Net Chunking Hackathon Hallucination Energy Hallucination Mitigation Hallucination Reduction HaluEval Hash-Verified State Haystack Health Systems Hierarchical Cognition Hierarchical JEPA Hierarchical Memory Hierarchical Memory Systems Hierarchical Reasoning High-Trust AI Systems House-Price-to-Income Household Debt Household Formation Household Fragility Housing Access Housing Completions Housing Crisis Housing Supply Hrm HRM Architecture Hugging Face Human Agency Human Feedback Human Value Human-AI Collaboration Human-AI Interaction Hybrid AI Systems Hydra Config ICL Scaffolding ICL Training Pipelines IFAC Image Generation Image Understanding In-Context Learning Inference Speed Inflation Infrastructure Capacity Inspectable AI Inspectable Models Int4 Int8 Intelligent Grep Intelligent Retraining Interest Burden Interest Rate Shock Interest Rates Interest vs Revenue Interest-to-Revenue Intergenerational Burden Intrinsic Hallucination Ireland Economy Irish Fiscal Policy Irish Housing Iterative Analysis Jitter Json Validation Knowledge Distillation Knowledge Graph Knowledge Graphs Knowledge Ingestion Knowledge Management Knowledge Measurement Knowledge Scoring Language Models Large Language Models Latent Variables Lean Leanstral Learning-From-Learning Least-Privilege AI LeCun Architecture Legacy Debt Lightweight AI Litellm Llama Factory Llama-Factory Quantization Llm LLM Agents LLM Alignment LLM Arbitration LLM Evaluation LLM Governance LLM Judgement LLM Pipeline LLM Routing LLM Supervision LLM Validation LLMs Local Models Logistic Regression Logits Long-Term AI Memory LoRD Distillation Low-Resource LLMs Machine Learning Macro Economics Macroeconomic Analysis Macroeconomic Modeling Managed Continuity MARS Master–pupil Architecture MCTS Measurement Measurement Crisis Memento Paper Memory Attribution Memory Composition Memory Diffing Memory Drift Memory Governance Memory Graph Memory Health Evaluation Memory Provenance Meta-Agents Meta-Learning Meta-Prompting Meta-Reasoning Metacognition in AI Metacognitive Awareness Mirror Machine Mixture of Experts ML Evaluation Model Alignment Model Architecture Model Comparison Model Compression Model Development Model Distillation Model Head Specialization Model Interpretability Model Monitoring Model Optimization Model Selection Model Switching Model Tuning Engines Model Validation Modified Domestic Demand Modular Embeddings Modular Scoring Moment Replay Monetary Policy Monotone Calibration Monte Carlo Tree Search Movie Generation MR.Q MRQ MRQ Evaluation MRQ Feedback Loops MRQ Models MRQ Scoring Multi-Agent Multi-Agent Systems Multi-Axis Evaluation Multi-Dimensional Scoring Multi-Model Reasoning Multi-Stage Reasoning Multi-Step Agents Multidimensional Scoring Multihead Attention Multimedia Multimodal AI Multimodal Representation Multinational Concentration Multistrategy Reasoning National Champions NER Retriever Nested Reasoning Loops Neural Network From Scratch Neural Networks NewsEdits Nexus Nlp Nn.Module NOVELSEEK Num_workers Object Detection Object Tracking Objective Function Observable Cognition OECD Pillar Two Ollama OOD Detection Open Source AI OpenCV Opinion Optimization Optimizers Orchestrator Ownership Access PACS Pairwise Evaluation PaperSearch Parameters Parkinsons Patient Support Pattern Detection Peft Per-Capita GDP Performance Permute Persistent Cognition Persistent Homology Perturbation Testing Pgvector Phos PHOS VPM Phōs PiHead Optimization Pin_memory Pipeline Learning Pipeline Optimization Pipeline Tuning Plan Traces Planner Executor PlanTrace Policy Policy Constraints Policy Learning Policy Networks Policy Refinement Policy Simulation Policy Synthesis Policy Trade-Offs Policy-Based Evaluation Policy-Bounded AI Pollinator Agent Population Growth Population-Capacity Gap Pose Estimation Postgres PostgreSQL Predictive World Models Preference Learning Preference Modeling Primitive Decomposition Primitive Reasoning Privacy-Preserving Search Procedural Learning Procedural Reasoning Productive Transfer Productivity Productivity Gap Productivity Growth Profiling Projection Geometry Prompt Compiler Prompt Engineering Prompt Optimization Prompt Tuning Proof Assistants Proof Search Provenance Enforcement Public Capital Public Investment Public-Service Capacity Pupil Models Python Python Ai Pytorch Q Learning Q Value Q-Learning With Embeddings Quality Model Quantization Techniques RAG RAG Verification Ranking Raspberry Pi AI Real Growth Real Wage Power Reasoning Reasoning Agents Reasoning Failure Analysis Reasoning Geometry Reasoning Models Reasoning Pathway Analysis Reasoning Provenance Reasoning Strategies Reasoning Systems Reasoning Trace Encoder Reasoning Traces Reasoning Verification Recurrent Neural Networks Recurrent Reasoning Blocks Recursive Models Recursive Search Recursive Self-Improvement Reference Gap Reference-Grounded AI Refinancing Risk Reflection Reflection Agents Reflective AI Regression Tuner Regressive Transfer Regularized Training Reinforcement Learning Reinforcement-Based Scoring Relevance Gap Reliable Revenue Rent Burden Repair Loops Replay vs Snapshot Equivalence Replayable Reasoning Reports Representation Learning Reproducibility Research Automation Research Filtering Research Reproducibility Reshape Retention Policy Retrieval Retrieval Is Not Proof Revenue Concentration Revenue Quality Reversible Systems Revision Reward Modeling Rhetoric RHLF RIVAL Framework Rmsnorm RMSNorm Stability Routing Rubric Classification Rubric-Based Evaluation Rule-Based Reasoning Runtime Epistemics Runtime Intelligence Runtime Search Sae Savings Resilience Scenario Analysis Scientific Research SCM Scoped Search Scorable Score Convergence Scoring Pipelines Scoring Stack Search Search as Cognition Search Biasing Search Self-Play Search Topology Self Consistency Self-Improvement Self-Improvement Loop Self-Improving Agents Self-Improving AI Self-Improving AI Systems Self-Improving Systems Self-Learning Self-Referential AI Self-Reflection Self-Reflective Agents Self-Supervised Learning Self-Tracking Self-Tuning AI Self-Tuning Loops Self-Tuning Systems Semantic Compression Semantic Drift Semantic Memory Semantic Retrieval Semantic Vector Tuning Sensitivity Analysis Sensitivity Score Shared Canonical Metrics Sharpening SICQL SICQL Integration SICQL Models Similarity Metrics SIS Small Model Tuning SmartAnswer Smolagents Snapshot Verification Software Architecture Sovereign Debt SpaceX Span Extraction Sparse Autoencoders Spatial Intelligence Sqlite SSP Stable Diffusion State Capacity State Transition Systems State_dict Statistics Canada Stephanie Stephanie AI Stephanie Framework Stepwise Reasoning Stochastic Generation Stock Market Stopping Conditions Strategy Evolution Strategy Memory Strategy Refinement Strict Attributed Accuracy Structural Deficit Structured Outputs Structured Prediction Structured Reasoning Structured Thinking Subconscious AI Systems Summarization Evaluation Superintelligence SVM Evaluation SVM Validation Symbolic AI Symbolic Control Symbolic Learning Symbolic Reasoning Symbolic Representation Symbolic Tuning Symptom Tracking Synthetic Content Synthetic Reasoning System Learning Systemic Risk Systems Analysis Systems Modeling Tax Receipts Tech Culture Technological Change Technology Tensor Shapes Test-Time Compute Testing Text Theorem Proving Theorems Thinking AI Tiny Critic Tiny Recursion Model Tool Calling Tool Loops Tool Use Tools Torch.compile Trace Learning Trading Training Transformers Tree of Thoughts Tree Search Tree-GRPO Trendslop Trustworthy AI Trustworthy RAG Uncertainty Uncertainty-Aware Reasoning Unsupervised Structure US Debt US Integration Valuation Pressure Value Capture Value Function Value Functions Vector Search Verbatim Evidence Verifiable AI Verification Verified Attribution Verifier Feedback Video Editing Video Processing View Visual Policy Maps Visual Primitives Visual Proof of Mind Visual Reasoning Visual Thought Maps Visualization VPM VPM-ViT Warranted Search Wearables WebUI Whisper Why Graphs Wikipedia Policy Windfall Tax Revenue Worldview Modeling Writer Writer Runtime Writing Tools XGBoost YOLO ZeroModel

Social

ernanhughes@gmail.com

© 2026 Programmer.ie: Modern AI programming.