Earlier this year I got interested in so called “dark agent factories” and tried to build one. I’m kinda centric among AI Hype and from my perspective coding definitely isn’t “solved” yet. But it’s also pretty obvious that AI agents can be super useful in day-to-day work. For sure for seasoned pros and not without concerns for laymans — but that’s a topic for another day.
AI Dark Factory I Aimed At
What I got interested in was possibility to throw tasks over a wall into an agentic factory with no supervision whatsoever. I wanted to cover a few pretty narrow cases:
- An unestablished codebase I don’t care much about
- An established codebase with some quality gates in place
- Getting agentic help with operations
Why it’s different from spinning agents via IDE or CLI on my work machine?
- I don’t setup anything for agents to do work
- Agents pick up work on their own
- Most importantly: agents don’t bother me with any follow ups they may have!
There’s still human-in-the-loop (HITL), though:
- For an unestablished codebase I just go & check if I like what agents did and maybe do several changes in one go
- For an established codebase I review PR / MR for which CI / CD pipeline is green
- For ops it was yet to discover
Development Approach
I was moving iteratively from running harness locally — I picked OpenCode out of the blue to replace it if needed but it survived — to running harness in Docker and finally running the whole factory in k8s cluster.
Key components I ended up with:
- Dev & ops pipelines: each is defined by own work items (WI) type, WI lifecycle and single AGENTS.md for step by step guidance
- Each WI lifecycle step is represented by a RQ queue & handler. The main thing a handler does is executing OpenCode in run mode via subprocess API. There’s also related work in there which makes sense to implement as imperative code.
- OpenCode works with WIs via dedicated MCP which supports different backends starting from an in-memory backend for local debug. Current prod backend uses task tracker but it’s not a strict requirement.
- In dev pipeline OpenCode interacts with GitLab either via git or API with proper credentials provided by the factory.
- In ops pipeline there’s a bunch of MCPs available to OpenCode — metrics & logs, traces is WIP — with permission checks in these MCPs. For ones requiring auth I got to set up proxying transparent to OpenCode. That way OpenCode doesn’t worry about auth when it uses these.
- AGENTS.md of repos agents work in may override pipeline guidance. Combining that with proper GitLab creds allowed MR-based HITL — and zero chance for agents to circumvent policy. Agent also considers any feedback provided in MR if WI is returned to previous lifecycle steps.
Results Produced by AI Dark Factory
I was impressed that I was able to cover all dev pipeline cases. And that Qwen3 under 30B was enough for that.
Ops experiments led me to the following workflow:
- Agent picks up an alert and analyze it.
- If it’s a noise it closes WI.
- If it’s not then it produces a mitigation plan.
- Anyways WI end up at HITL stage where dev work items should appear. These either aim at increasing signal / noise ratio or fix something.
- That makes ops nicely recurse into dev because modern teams should treat their ops stuff as code.
Closing Thoughts
With all these impressive results AI Agents Factory (or AIAF for short) is far from being a complete product, even for narrow cases it was designed for. A lot of work is ahead but it’s going to be an interesting journey. Hope to share more in the near future!