Building AI Agents That Actually Work

Andrew Luo Weimin

aiagentsproductivitylearning

345 Words 1 Minute, 34 Seconds

2026-05-23 11:31 +0000


Notes from Remy Gaskell’s YouTube Video (eA9Zf2-qYYM)

1. Chat vs. Agents

  • Chat Models: A simple ping-pong interaction (Question $\rightarrow$ Answer / Back-and-forth).
  • AI Agents: A goal-driven execution model (Goal $\rightarrow$ Result). You specify the outcome, and the agent plans and executes autonomously to achieve it.

2. The Agent Loop (Observe-Think-Act)

Agents operate in a continuous loop:

  1. Observe: Research, gather context, and analyze the task environment.
  2. Think: Plan the workflow, formulate steps, and evaluate options.
  3. Act: Execute actions (write code, run commands, fetch resources) and verify the outputs.

3. The “AI Operating System” Framework

An effective agent setup is treated like an Operating System, structured using specific configuration files and folders:

Essential Files

  • agents.md: The onboarding document for the AI. It defines the business voice, user context, expectations, and the agent’s role.
  • memory.md: A living file updated dynamically. It stores user preferences, rules of thumb, and learned behaviors (e.g., specific formatting rules, things to avoid) so the agent self-improves and becomes more personalized over time.

Structured Folders

  • Context Folder: Contains rich reference documents, templates, and deep background material relevant to the project or business.
  • Skills Folder: Reusable Standard Operating Procedures (SOPs) for the AI. Once a complex workflow is solved, package it into a “skill” (a structured prompt/script) so the agent can execute it autonomously in the future.

4. Key Technical Concepts

  • Agent Harnesses: Platforms like Claude Code, Antigravity, Codex, Cowork, and Manus act as the “chassis/car,” while the LLM acts as the “engine.” Learning how the agent loop works in one tool allows you to operate any of them.
  • MCP (Model Context Protocol): An open standard created by Anthropic that acts as a translator, allowing LLMs to safely connect to external tools (Gmail, Notion, Slack, databases, browsers) without custom development.

5. Compounding Productivity

  • Automate SOPs: Shift from manual administrative work to building automated skills. Once a skill is written, that task is automated forever.
  • The Math of Compounding: Automating 3–5 small tasks a week builds massive leverage, eventually allowing individuals to run entire automated departments and compress 40 hours of work into a single afternoon.