What's been cooking — March 2026

8 merged PRs across 5 repos

What's been cooking — March 2026

March was the month of "make it go faster" for the GLiNER family, with quantization, torch.compile, and a pile of vectorization landing across both forks. Meanwhile, the Exa side of the house spent its energy on the unglamorous-but-necessary work of plumbing a new dashboardUrl field through every layer of the stack.

fastino-ai/GLiNER2

The headline act was a serious GPU performance push. GLiNER2#76 rewrote span representation to run as a single batched forward pass instead of N calls into SpanMarkerV0, replaced an O(text_length * max_width) Python loop with torch.arange and broadcasting, and switched to gather-based embedding extraction — netting a 22–57% inference speedup. GLiNER2#86 then added model.quantize() (fp16 via model.half() with dtype-aware allocation throughout the forward path) and model.compile() (torch.compile(dynamic=True) on the encoder and span-rep core, leaving the per-sample decode eager because of an unavoidable graph break), pushing speedups to 2.42x. Finally, GLiNER2#88 hunted down a graph break in count_embed by replacing nn.GRU — which dynamo simply can't trace — with a CompileSafeGRU that implements the cell equations using F.linear, recovering another 11–44%.

urchade/GLiNER

The upstream repo got a port of the same treatment in GLiNER#342: a quantize parameter on from_pretrained() and from_config() that accepts True/"fp16" or "bf16", plus a compile() that actually works on variable-length NER inputs thanks to dynamic=True and capture_scalar_outputs=True. About 1.9x on GPU, which is roughly what you'd hope for from fp16 alone.

exa-labs/exa-py

Two small but correct changes. exa-py#184 retypes employees on WebsetItemCompanyPropertiesFields from Optional[float] to Optional[int]. exa-py#183 promotes dashboard_url and title to first-class typed fields on the Webset model so they stop falling through into model_extra.

exa-labs/exa-js

exa-js#147 mirrors the Python change, adding the required dashboardUrl string to the Webset OpenAPI type so JS consumers can reach it without casting through any.

exa-labs/websets-mcp-server

websets-mcp-server#13 closes the loop by adding dashboardUrl to the Webset TypeScript interface and updating the create_webset, get_webset, and list_websets tool descriptions so the model on the other end actually knows the field exists and can surface it to users.

That's the month: GLiNER got noticeably faster, and a small URL field got the full multi-repo rollout it deserved.