AI-driven parametric 3D model generator

I spent a month and a fair amount of lost sleep building a parametric 3D model generator, where an AI agent gets a prompt like “simple bookshelf” and spits out a ready STEP, GLB and FCStd file. It works. And I still haven’t generated anything I couldn’t do by hand in FreeCAD GUI in five minutes.

This is an honest write-up of a project running under the codename bluecad. I don’t have a final name yet. I’m also not publishing the source for now, because I’m not sure which direction I want to take it.

What it was supposed to do

The idea was simple. You type “bookshelf with five shelves, 180cm tall, 30cm deep” in chat, the agent understands the geometry, plans features the way an engineer would in PartDesign (body → sketch → pad → pocket → fillet), and the backend produces a parametric FCStd file plus STEP, STL, GLB, OBJ and 3MF exports.

The stack is fairly conventional:

• Monorepo on Turbo + pnpm
• API in NestJS, BullMQ, Drizzle, Supabase auth
• Worker in NestJS, consuming the job queue
• FreeCAD running as a separate Python service (FastAPI), because FreeCAD has no usable API in any language other than its own Python
• Web in Nuxt 4 and Pinia
• Postgres, Redis, Supabase Storage for files

The frontend (Nuxt 4) has a job queue view, organizations, projects, and a separate 3D viewer for the generated GLBs.

Why FreeCAD

I wanted real BREPs, not meshes. The goal was that at the end of the pipeline an engineer could open the generated file in stock FreeCAD, see the PartDesign tree, change a pad length, move a sketch, add a feature, recompute. A mesh wouldn’t give me that. STL out of a generative tool is a dead end.

That narrowed the list fast. FreeCAD is open source, runs headless, has parametric bodies and the classic PartDesign workflow that anyone who has touched a commercial CAD knows. Alternatives ruled themselves out quickly: OpenCascade directly is too low-level and produces no editable tree, Blender is great with meshes and poor at BREPs, and commercial options like OnShape or Fusion come with closed APIs and license costs.

The choice was easy. Later I deeply regretted it.

JSONCAD, the layer the agent has to understand

The agent doesn’t generate Python. It generates an intermediate layer I called JSONCAD. It’s a declarative description of operations that I can validate, replay, version, and most importantly: hand to an LLM in a simple shape.

Example: “cube with a hole”:

1
{
2
  "operations": [
3
    { "type": "body", "id": "body1" },
4
    {
5
      "type": "sketch",
6
      "id": "sketch1",
7
      "parent": "body1",
8
      "plane": { "position": [0, 0, 0], "rotation": [0, 0, 0] },
9
      "entities": [{ "kind": "circle", "center": [0, 0], "radius": 5 }]
10
    },
11
    {
12
      "type": "pad",
13
      "id": "pad_0",
14
      "parent": "body1",
15
      "profile": "sketch1",
16
      "extent": "distance",
17
      "length": 20
18
    }
19
  ]
20
}

Every operation has its own zod schema and a mapping to FreeCAD’s newObject. The worker takes JSONCAD, calls the Python service, which runs the right handlers and a recompute() on the document. All the agent has to keep in its head is a dictionary of a dozen-something operation types.

Why I didn’t give the agent JSON directly

The first version forced the agent to return JSON. Hit rate was mediocre. LLMs hallucinate fields, confuse id with name, drop the odd bracket, sometimes start wrapping JSON in markdown. The validator was throwing out 30-40% of responses.

I switched to a TypeScript-flavored DSL. The agent emits something like:

1
cad.body('body1')
2

3
cad.sketchPlane('sketch1', 0, 0, 0, 0, 0, 0)
4
cad.line(0, 0, 100, 0)
5
cad.line(100, 0, 100, 100)
6
cad.line(100, 100, 0, 100)
7
cad.line(0, 100, 0, 0)
8
cad.padDistance(10)
9

10
cad.sketchPlane('sketch2', 0, 0, 10, 0, 0, 0)
11
cad.circle(50, 50, 10)
12
cad.pocketDistance(10)
13

14
cad.filletAll(2)

Under the hood it’s still JSONCAD, just built by a fluent builder. The system prompt says it plainly: the first line must start with cad., no const, no let, no loops, no conditionals, one line per method call. DSL validation is brutally simple:

1
export function validateDsl(code: string): boolean {
2
  for (const raw of code.split('\n')) {
3
    const line = raw.trim();
4
    if (!line) continue;
5
    if (!line.startsWith('cad.')) return false;
6
    if (!line.endsWith(')') && !line.endsWith(');')) return false;
7
    if (/[{}=]/.test(line)) return false;
8
    // method name has to be on the allowed list
9
  }
10
  return true;
11
}

After this change the generation success rate jumped past 90%. Same prompt, same Claude on the backend, only the shape of the response changed. Constraining the agent worked better than teaching it the format.

The FreeCAD API is pain

I expected this to take a few afternoons. It took weeks.

FreeCAD has multiple API layers that don’t fully agree with each other. There’s Part (low-level geometry), PartDesign (parametric features), Sketcher (2D sketches), Draft. Each set of docs points you in a different direction. What works in Part doesn’t necessarily survive being placed inside a PartDesign::Body. Some operations require recompute() at a very specific moment, others corrupt the document if you recompute() too early.

A classic: fillet. In the GUI you click edges. In the API you supply a list of names like Edge12, Edge17. Those names shift on every model change, because FreeCAD has no stable TNP (Topological Naming Problem is a well-known, open issue and has been for years). I ended up writing edge selection by geometry instead of by ID:

1
for i, edge in enumerate(shape.Edges):
2
    v1, v2 = edge.Vertexes[0].Point, edge.Vertexes[1].Point
3
    dx, dy, dz = v2.x - v1.x, v2.y - v1.y, v2.z - v1.z
4

5
    if kind == "vertical":
6
        if abs(dx) < tol and abs(dy) < tol and abs(dz) > tol:
7
            edges_to_use.append(f"Edge{i+1}")
8
    elif kind == "axis" and selector.axis == "z":
9
        if abs(dx) < tol and abs(dy) < tol:
10
            edges_to_use.append(f"Edge{i+1}")
11
    # ...

Instead of “take edge number X”, the agent says “take all vertical edges” or “all outer edges”. It works with concepts, not identifiers. One of the few pieces of this project I’m actually happy with.

What I managed to generate

This is where things get less flattering.

A cube with a hole. Looks fine. I was trying to get “a cube with two intersecting holes”. After several turns the agent insists there are two. The model still has one. The second pocket was hitting the wrong plane or wasn’t being registered as part of the body.

A bookshelf. Prompt: “simple bookshelf”. The feature list matches what the agent wrote itself (“rectangular frame, vertical side panels, five evenly spaced shelves”): the frame is there, walls are there, five partitions too. Except the shape lies on its side, it’s wider than tall, and the “shelves” stand vertically instead of horizontally. It came out as a cutlery tray, not a bookshelf. Formally matches the spec, useful for absolutely nothing.

A table. Prompt: “generate simple circle table with 6 legs”. First attempt: a tabletop with six holes through it, zero legs. After a follow-up “there isn’t legs” the agent added legs but the holes in the top stayed. Anything a normal user would call a table is still an abstraction here.

Trade-offs and what doesn’t work

Full disclosure of what went badly:

No feedback loop. The agent emits DSL, the validator checks syntax, FreeCAD executes. Nothing verifies “render it, look at it, fix it”. Without that loop the model has no idea whether the bookshelf stands vertically, whether the “shelves” are oriented horizontally, or whether the proportions match anything you’d actually put a book on.

No geometric memory. The agent doesn’t know where the previous pad ended, so the next sketches either land on random planes or fall back to simplifications (mostly XY at Z=0). Every model comes out “flat” in a compositional sense.

Topological Naming Problem. Geometry-based edge selection saves simple fillets, but only for simple bodies. With complex geometry (loft, pipe, mirror) there’s no clean way to refer to an edge produced by two earlier operations.

Headless FreeCAD likes to crash. Certain boolean + fillet combinations take down the whole FreeCAD process, not just return an error. The worker needs a health check and a restart.

Token cost. Longer prompts with the full DSL description and workflow constraints come out to 3-4k input tokens plus the response. With “fix this” iterations it gets expensive.

Why I’m keeping the source closed

I’m not publishing the code right now. Two reasons.

First, I don’t know where this project is going. I have several directions on the table: an agent with a closed render-and-critic loop (a multimodal model looks at the GLB and corrects itself), a generator for a narrow domain (3D printer mounts, where the design space is finite), or simply a better DSL with stronger primitives. Each path needs a different architecture. Opening source for code that could flip 180 degrees doesn’t make sense.

Second, in its current state the code is more of a proof that the problem is harder than it looked than a working tool. I’d rather not encourage anyone to fork something that can’t draw a table with six legs.

What I’d do differently

I’d seriously consider CadQuery or build123d instead of FreeCAD. Both sit on OpenCascade and produce honest BREPs you can export to STEP, just without FreeCAD’s object layer. The trade-off is real though: I lose the FCStd file with a PartDesign tree you can edit in the GUI. STEP opens in FreeCAD, but as a single solid, not a list of features you can tweak. The question is whether that editability is actually valuable to the user or just looks nice on a slide.

I’d start with a critic loop on day one. Render the result to PNG, multimodal model compares against the prompt, next iteration. Without that the agent is shooting blind.

And I’d go narrow, not general. A generator for “any 3D model” isn’t a single winter’s work. A generator for GoPro mounts is two weeks and actually works.

When this makes sense

The idea of an LLM as a CAD interface makes sense, just probably not in the shape I started with. Right now it looks reasonable for:

• very closed domains (parametric components, mounts, fixtures),
• generating CadQuery code directly instead of a layer above it,
• tooling like “sketch autocomplete” beside an engineer, not replacing them.

I’m coming back to this project, probably with a different backend and a much narrower scope. When there’s a final name, I’ll write about it.