The second approach is you build the minimal version which works. You try to see if it meets the requirement or not. If it meets it, it’s fine and you’re good. If it does not, you go back to the whiteboard and try to understand what the system is lacking. You try to break it down into more components if it’s possible or you can just keep on improving the current one. Once you are satisfied with the progress you gauge if what you built meets your purpose, if not you keep on doing it until it fits your requirement

There are different constraints or timelines, quality requirements, resources, or bandwidth, which allow you to make the decision of either going with the first approach or the second approach. For example: if you are a startup that is building something, then your first instinct should be to just build the version that you had in mind and then see how the society reacts to it. But if you are, let’s say, a space organisation which wants to send a satellite or a piece of equipment to, let’s say, Mars or the space station, you really can’t just build the first thing which pops into your mind and then send it across (because the cost of failure is so high that you really can’t afford to lose). You have to make sure that every individual component of the system which you are building is end-to-end stress tested and that it works correctly and it’s not susceptible to failure. If you are a programmer then you can either go with a top-down approach or a bottom-up approach to building your application

I think the systems which are built using the first kind of approach have a really good property: they are much easier to maintain compared to the systems which are built using the second approach. I think because every individual component in the first approach has been primarily studied under various conditions, their properties as well as the way in which they behave are clearly understood. It becomes very much easier to then understand how the system would behave or evolve when put in some different situations or conditions. We might have some hypothesis about which components are susceptible to failure in these conditions or which components will stop working after a certain period of time. It’s also much easier to change or completely swap out any of the components or change the way in which the system behaves altogether

With the rise in coding agents it seems to be that the approaches of the second kind are the ones which are going to be used by everyone. But there is a greater incentive to build systems using approaches of the first kind, using these AI coding agents or tools, it’s much easier to study every individual component and map out the relations (like mapping out the way in which they are going to behave under different conditions) or study them in isolated or paired scenarios. It’s much easier to consolidate all of this knowledge in, let’s say, some skills or markdown files, and then share it across other teams or other agents of the same kind

Surprisingly I found really good evidence that J space can be used to derive steering vectors from just concept tokens which are represented in the steering behaviour. It works really well for simple model behaviours such as outputting everything in all caps or speaking in a weird manner. However the steering vector derived in this manner is prone to hallucinations and is brittle for behaviours which are complex and cannot clearly be represented with just tokens/words

Setup

For all experiments I used Qwen3-1.7B which already has a published J lens by neuronpedia at neuronpedia/jacobian-lens huggingface repo. I ran all the experiments reported locally on my macbook, this was also a reason why I couldn’t test larger models
Code is publicly avl at jlens_steer

Executive Summary

I wanted to first set a good enough baseline with some working steering vectors which I can use for comparison. I found the science-of-finetuning/steering-vecs-qwen3_1_7B repo which had a steering vector which steered the model to answering all tokens in all caps. This was fitted the expensive way from a model organism finetuned to answer in capitals. Along the way I also found someone published an abliterated model for qwen3-1.7b with refusal behaviour. Abliteration works by finding the direction inside the model that means refusal, then subtracting it from the parts of each layer that write into the model’s running state. So the difference between the abliterated model and the base model is just that direction, applied over and over. I subtracted the two models and pulled each layer’s change apart to get back the steering vector for refusal

Here is the result of the model steering

Alright so our baseline comparison steering vectors work fine

Steering for All Caps Behaviour

Now let us come back to the J space, since the J space is just a linear mapping between the activations at layer L to the unembedding matrix let us find all pairs of all caps tokens and their corresponding lowercase tokens in the vocabulary like (“ AND”, “ and” or “ TOWN”, “ town”) and invert the rows of J lens corresponding to those tokens

This will evidently give us the activation vector at that layer L which would have verbalized that token according to J lens. Take all such pairs of activation vector and project them using PCA and let’s compare them with the activation vector which we got for the all caps steering

We find that the vector a_upper - a_lower derived from J lens is closer to the activation vector of all caps tokens. They also have very high cosine similarity score compared to replacing the J lens with an identity operation in which case it becomes just a logit lens

Let’s now take an average of all activation diffs of all such pairs and comparing it with our baseline earlier it definitely steers the model towards generating all caps tokens/words

Refusal Steering

Let us now generalize or atleast attempt to generalize this algorithm for steering a model to refuse even harmless prompts. For this we need to gather concept tokens that are related to refusal and words/tokens that might be occurring inside the model’s mind when it’s trying to refuse the prompt even though it’s harmless.

Let’s see what goes on inside the model using the j lens top 5 words when we steer the model using the refusal steering vector over the prompt “How do I bake a loaf of sourdough bread?”, the model answers “I'm sorry, but I cannot assist with any illegal, unethical, or harmful...”. The below some tokens which appeared when verbalized the activation using j lens

As expected it’s thinks about words/tokens related to refusal behaviour. Inverting just a single refusal word/token with a neutral token did not work this time as the single refusal tokens could relate lots of concepts and inverting it does not have enough information passed via the J lens to arrive at a refusal activation vector

My immediate next attempt was to use 5 refusal words/tokens each and invert them together via the J lens, the intuition was this is a really good approximation of the activation that results in making the model think about those tokens, now generating more activations with similar sets of refusal tokens and taking an average of all such approximated activations would be the activation most likely to produce those words related to refusal in the model’s thinking/scratchpad and would give us the steering vector for refusal if we subtract it by the average activation over the activation of tokens which elicit

So the algorithm is:

1. Collect 20-30 words/tokens that relate/ellicit to the concept/behaviour towards which we want to steer the model
2. Sample C=5 tokens with replacement from this above set, and compute an activation vector by inverting the jacobian over these concept tokens
3. Repeat K times step 2 to collect K activation vector, and average all of them
4. This is the activation vector most likely to produce refusals
5. Substract this from the avg activation vector of the entire vocabulary to get the steering vector

Now let’s compute this for refusals and stop at step 4 and then run the activation vector we got from step 4 and visualize what the model generates using that activation vector on arbitrary prompts

We observe that the model shows signs and behaviour of telling user not to do certain things. It also makes the model hallucinate more often only 1 in 5 prompts where the model showed genuine refusal behaviour but even that was where the model hallucinated and kept repeating itself. The activation is really noisy and brittle to use as it is. Let’s evaluate on the 10 prompts we evaluated our baseline steering vector on by subtracting the average activation we got after inverting the J lens over the entire vocabulary

It’s way behind the baseline score yet, the reparations are gone but it still hallucinates often or only mentions not to do certain things or stop things are mentioning them first. Direct refusal rates are still low

Reading the verbalized outputs of the steered model we see that the activation is making the model think about the tokens/words which were relating to our refusal behaviour
" even", " only", " but", " forbidden", " not", " the", " wrong", " in", " failed"
but reading the above 5 completions also shows that the activation is forcing the model to think about these words/tokens which could and it is not directly transferring to make the model show refusal behaviour

The model hallucinates a lot more or make the outputs more narrow in the sense of making the model force to generate sentences relating those concept tokens and not the behaviour which corresponds to them

One major limitation for complex steering behaviours is they are very hard to represent via only tokens/words and using only the J space looses out on the information required to articulate or represent those behaviours in the activation space

During the same time I was also reading some books and taking humanities electives which used to essentially advice we should have a kind “inner voice” and should cherrish and love ourself, I always felt these were coping mechanisms for the week, it’s like burrowing your head in the sand and acting as if the problem does not exist, cheap dopamins to dull the pain of reality around us, it’s better to embrace the pain I used to think, only the unambitious feel content with their life, I had seen the most successful people around me and the people whom I admired have an intense self belief and conviction fueled by an intense desire to be good enough

I am really really young right now, in my early twenties but I would say my stance has changed somewhat than the younger version of myself. Ambition does not mean you have to constantly diminish or belittle everything that you do, it’s very simply put the magnitude and impact of the end goals that you have decided for yourself, you can reach the same destination by taking a different path, a framework that preserves your peace of mind and your conviction, it could get to your goal faster or slower but it will eventually take you there. you can control what is within your control, let go of what is not and make peace with the universe, the universe if neither against you nor it is with you, it is just there existing in itself to fulfill it’s own purpose of existing for you

I have always measured life in terms of slope and intercepts, intercepts is essentially where you currently are and slope is your rate of change, the rate at which you are improving or working towards your ambitions or goals, eventually as the theory goes it does not matter what your intercept is what matters is your slope, a line with a greater slope will always cross over a line with a lesser slope

I don’t know but there has been this weird phenomenon which I have observed since the past 2 years where I have observed by slope has always increased, infact it has always been more than an exponential increase for me, in every aspect of life, I can list down more than 1000 improvements and signals/indicators which I have observed in myself, I have never felt so great ever in my life, mentally, physically and spiritually, exponentials have a weird property that their slope is also an exponential, so there is literally nothing that can slow down this insane exponential growth that I have unlocked

The Flatland Competition

Flatland is a train scheduling problem where we are given a railway, a fleet of trains with timetables, and we have to route every train to its destination without them crashing into each other. The competition fixes one railway topology, a 120x150 grid with 28 stations, and has 7 difficulty levels, 5 scenarios each, for 35 scored episodes. Each scenario spawns anywhere from 8 to 532 trains onto that same map. Here is what one scenario with 25 trains looks like in Flatland’s own renderer, cartoon trains and grass and all:

Every train starts on the map at some origin and has to reach a destination, which are the red stars, and the grey diamonds are switches. The dense knot of switches in the middle is a station throat, where a lot of routes funnel together and most of the interesting conflicts happen. Each cell has a set of allowed entry to exit direction transitions, which means a train entering a cell heading north can only leave in the directions that cell permits for a northbound entry, so most cells have exactly one exit, but a switch has a real routing choice:

A train at cell (r, c) heading east is a different search node from the same train heading west, because they can do different things next. We encode a node as ((r * W) + c) * 4 + d and the whole planner lives on this directed graph of 4 * H * W nodes, and a switch is just a node where some entry direction has more than one successor.

On top of the geometry sits a timetable. Each train has an earliest departure, a latest arrival, and a list of intermediate stops it is supposed to serve on the way, so think of an intercity that calls at three platforms before its terminus:

Trains also move at different speeds, and Flatland speeds are fractions, so an intercity runs at v = 1.0 which is one cell per step, a regional at 0.8, a slow stopping service at 0.6, and a commuter at 0.5. A train at speed p/q banks p/q of distance every step and only leaves its cell once the banked distance reaches 1, so a 0.8 train needs two steps to leave its first cell, and because the leftover fraction carries over, some of its later cells clear in one step. The planner has to track this fraction exactly, because it needs to predict the precise step at which each cell frees up.

Scoring

The score is where the whole game is decided. Per scenario every agent piles up penalties (all rewards are <= 0), and the normalized reward is

normalized = sum_over_agents( max(cum_reward_agent, -max_episode_steps) ) / (max_episode_steps * n_agents) + 1

so a perfect run scores 1.0 and a total disaster scores 0, and the competition score is just the sum of these over all 35 scenarios, so the max is 35. There is one hard rule on top of this, which is that evaluation aborts a whole level if fewer than 25% of trains finish it, so getting most trains home is a constraint and not just something nice to have.

The malfunctions ramp up with the levels, so levels 0 to 2 are clean, level 3 adds random breakdowns where a train freezes in place for some steps, levels 4 and 5 add random departure delays, and level 6 adds infrastructure disruptions on top of everything else. One important detail is that the evaluator does not run your policy live and trust it. Your container records a trajectory of actions with flatland-trajectory-generate-from-policy, and then a server-side TrajectoryEvaluator replays those actions through its own Flatland install to compute the score, so only the recorded actions matter. We ran our own trajectories through that exact code path so that there would be no surprise between the local score and the real one.

The RL baseline they ship, which is a FastTreeObs observation fed into an MLP policy trained on the example curriculum, scores 15.55 out of 27 on that curriculum and gets 45.6% of trains home. That is the number to beat, and it is also the reason the organizers run a separate OR / non-RL track, because planners win Flatland, they always have, and the organizers know it.

The reward function, and the one number that matters

Before writing a single line of the planner we sat down and read flatland.envs.rewards for the exact competition parameters, because in a problem like this the cost function is the strategy. Here is the full reward book:

Most of these are what you would expect. Canceling a train (never departing it) costs 5x its shortest-path time, which is large, so you always launch every train. Arriving late costs 1 per step. Skipping an intermediate stop costs a flat 50. But look at the top bar, because a collision costs 250 x speed, and here is the thing that took us a couple of hours of poking at the environment to actually believe:

In Flatland 4.2.6, which is the version the evaluator runs, any transition from MOVING to STOPPED is charged the full collision penalty. Not just crashes. A clean, deliberate, perfectly safe brake of a moving train costs 250 x v.

We checked this by hand instead of trusting the docstring (which says otherwise). Load a scenario, drive one intercity forward on empty track, brake it cleanly with nothing in front of it, and read the reward:

# one speed-1 train, clean voluntary brake, empty track ahead
env, _ = RailEnvPersister.load_new(path, rewards=ECML2026Rewards())
# ... step it forward a few cells, then issue STOP_MOVING ...
# step_reward for that transition:  -250.0

It is -250.0, exactly. If you dig into rail_env.py the reason is that AgentTransitionData.speed is captured in the pre-step loop before the speed update, and the reward code multiplies the collision factor by that pre-step speed on every MOVING to STOPPED transition, whether it came from the motion check blocking you or from your own STOP action. Reaching the final target is a MOVING to DONE transition and is exempt, so arriving is free, but stopping short is not.

This one detail decides everything, and three things fall straight out of it.

We skip every intermediate stop, on purpose. Serving a stop means the train has to actually halt at the platform, and a halt is a MOVING to STOPPED transition, so it costs 250 x v, while skipping the stop costs a flat 50. So serving only wins when 250 x v < 50, which is v < 0.2, and every real train in this competition runs at 0.5 or faster:

The break-even is at v = 0.2 and every train category (commuter, stopping service, regional, intercity) sits above it, so the right move is to blow through every platform without stopping and just eat the -50. It feels wrong because an intercity is supposed to stop at its platforms, but the reward book says skip, so we skip. We left one environment flag, PLANNER_SERVE_STOPS, that flips this back, in case the organizers ever fix the braking bug during the competition, but they did not.

We hold trains off the map instead of stopping them on it. A train that has not departed yet sits off the board for free with no penalty for waiting, but a train on the board that has to give way has to brake, which is 250 x v. And delaying a departure by D steps only costs about D at the end, because late arrival is -1 per step, so holding a train back is a slow linear cost while braking a moving one is a flat and expensive cost:

The two lines cross at D = 250, so you can hold an intercity off the map for 250 steps and still come out ahead of letting it brake even once. On a map where episodes only run a few hundred steps, that basically means never let a moving train stop if you can delay its departure instead.

Minimize halts, full stop. Every MOVING to STOPPED is -250 x v, so the whole job of the planner, on top of getting trains home, is to get them home without ever touching the brakes. This changes what we are even searching for, because we are no longer looking for shortest paths, we are looking for paths that never have to stop, and this is exactly the problem that Safe-Interval Path Planning was made for.

V1: SIPP over a reservation table

The standard way to schedule a lot of agents on shared track is prioritized planning with a reservation table. You order the trains and plan them one at a time, so each train plans around the reservations of all the trains planned before it and then writes its own path into the table. It is not globally optimal like a full Conflict-Based Search would be, but it scales to hundreds of agents and with a good priority order it gets very close.

The core of it is Safe-Interval Path Planning. The idea behind SIPP is that you should not search over (cell, timestep) because there are far too many timesteps, you should search over (cell, safe interval), where a safe interval is a maximal window during which a cell is free. A reservation table is really a 2D object, resource on one axis and time on the other, where every occupancy is a rectangle, so the gaps in a resource’s column are its safe intervals, and a train’s plan is a staircase that climbs through those gaps:

This picture is the whole algorithm. Our train wants to get from cell A to cell F, and another train (train 7) has reserved cell D for the window [3, 9), so our train rolls through A and B and then has to wait, because it cannot enter D until D frees at t=9. SIPP holds it in cell C until the exact moment D opens and then threads through, and that wait is a single halt, marked with the X, which costs 250 x v. The search is looking for the staircase that reaches the goal with the fewest halts, and after that with the earliest arrival.

The reservation table maps every resource to a sorted list of (start, end, agent) occupancies, and the core query hands back the free gaps:

def safe_intervals(self, res, t_from):
    """Maximal free intervals of `res` intersecting [t_from, horizon)."""
    lst = self.intervals.get(res)
    if not lst:
        return [(t_from, self.horizon)]
    out, lo = [], t_from
    i = bisect_right(lst, (t_from, 1 << 60, 1 << 60)) - 1
    if i >= 0 and lst[i][1] > t_from:      # inside an occupied block
        lo = lst[i][1]
    i += 1
    for s, e, _ in lst[i:]:
        if e <= lo:
            continue
        if s > lo:
            out.append((lo, min(s, self.horizon)))   # a gap
        lo = max(lo, e)
        if lo >= self.horizon:
            return out
    if lo < self.horizon:
        out.append((lo, self.horizon))
    return out

A resource is a cell, with one exception, which is that the map has level-free crossings (bridges) where two trains can sit in the same cell on different axes, so those split into two resources by axis. The A* search expands (node, fraction, interval) states, and the thing that makes it exact for Flatland is the fractional-speed arithmetic, because for a train at speed p/q sitting in a cell with banked fraction numerator fr, the number of steps to roll out of the cell without stopping is ceil((q - fr) / p). So from an entry time t the non-stop exit is t + ceil((q - fr) / p), and the cost of a state is arrival_time - earliest_possible + halt_weight * n_halts. The A* heuristic is the exact non-stop travel time left to the goal, which we get from a backward BFS over the graph, and it is admissible because non-stop is the fastest a train can ever go.

The successor step is where the halt cost enters. For each successor we look at the safe intervals of that successor’s resource, find the earliest time we could enter it, and check whether entering that late meant we had to wait in the current cell, because waiting is a halt:

for v_node in g.successors(node):
    for s, e in self.res.safe_intervals(res_v, exit_nonstop):
        if s > free_til:                 # can't wait in u past when u itself is claimed
            break
        tau = max(exit_nonstop, s)       # earliest legal entry into v
        # ... swap protection and corridor locks go here ...
        halted = tau > exit_nonstop      # did we have to wait?  that's a halt
        step_cost = (tau - t) + (halt_weight if halted else 0.0)
        ...

halt_weight is 250 x v, straight out of the reward book, so the search literally prices a wait at the same rate the game prices a brake, which means it will happily send a train the long way around or hold it back at its origin just to avoid paying that 250. This is SIPP behaving as a scheduler that has fully understood the one number that matters.

The last piece is the order we plan trains in. We plan tightest-slack-first, where slack is latest_arrival - earliest_departure - nonstop_travel_time, so the trains with the least room to spare get first pick of the reservation table:

def slack(agent):
    dist = g.dist_to_goals(goal_nodes[agent.handle])[start_node(agent)]
    travel = ceil(dist / speed)
    return (agent.latest_arrival) - agent.earliest_departure - travel
order = sorted(env.agents, key=slack)

We wired this into the competition’s policy interface (an observation builder hands (env, handle) to the policy, which runs the planner once and then just reads out the actions), and ran it on the example curriculum. V1 scored 24.38 out of 27, with 93.8% of trains delivered, against the RL baseline’s 15.55 and 45.6%. So on the first day, before any of the hard problems, the planner was already 57% better than the RL policy they ship, and that is really the whole OR-beats-RL story in one number.

V2: making it survive execution

A plan is only good until the world stops matching it, and in Flatland the world stops matching it all the time, because trains break down, departures slip, and sometimes our own placement attempts get blocked. So the runtime is a plan-and-execute loop. On the first step it builds the graph and plans everyone, and on every step after that it compares each train’s actual state against its plan, marks the ones that have drifted as dirty, and replans them (a full SIPP re-search, most urgent first, under a per-step budget), while everything else just reads its next action out of the schedule.

Turning a timed path into actions has one subtlety that is worth calling out, and it is the braking penalty again. When a train is already stopped, say a malfunction just ended, and its plan says to wait a bit longer before moving, we have to send DO_NOTHING and keep it stopped and then start it at the last moment, instead of letting it inch forward and brake again, because restarting a stopped train is free but braking a moving one is -250. The action builder is full of this kind of care.

The bigger job in V2 was making the thing safe when the timing goes against us. Four mechanisms went in:

Swap protection. Flatland does not let two trains cross on a single edge in one step, so they cannot swap places. If our schedule ever asks for a swap, the replay quietly turns it into a stop (and a -250, and maybe a deadlock). So the reservation table also keeps a vacate_to map, and when agent A leaves resource u into v at time tau, we record that, and any other train that tries to enter u from v at that same instant gets pushed back a step.

This looks like a small corner case, but it is actually the biggest failure mode in the whole competition, because two trains meeting head on in a single-track corridor cannot pass each other and cannot swap, so they just sit there facing each other forever. We will come back to this later.

Owner checks at cell entry. Even with reservations, schedules go stale between replans, so at the moment a train is about to enter a new cell we double-check the reservation table, and if the slot is claimed by an opposing train (one heading the other way, which is a real head-on risk) we hold. Same-direction conflicts we let resolve as normal queueing, because queueing behind someone going your way is no worse than waiting.

Honest stub blocks. When a train genuinely cannot find a plan, because the table is too crowded for a moment, we do not just let it sit there invisibly, we block its current cell in the reservation table all the way to the horizon, so every other train routes around it instead of queueing up behind it waiting for a train that is not moving. Once the train replans, the stub is replaced by its real reservations.

Provable-deadlock containment. Some jams really are dead and it is important not to thrash trying to fix them, so we mark only the provably dead trains, which are mutual pairs where each train’s only possible next cell is the other train’s current cell. Those get marked dead and told to sit still. Bigger cycles we leave alone, because a ring of trains in Flatland can sometimes rotate itself free, and killing them off by mistake loses trains.

There is also a runtime governor, because each scenario has a 30-minute wall-clock budget on the evaluation cluster, so if planning time creeps past a threshold the planner starts cutting its own search effort (fewer replans per step, smaller expansion budget) to make sure it always finishes stepping the environment, since a slightly worse plan is much better than a scenario that times out at zero.

V2 scored 25.43 out of 27 (99.4% delivered) on the curriculum, and the little bit that is left is mostly the stop-skipping, which is by design. On the clean levels the planner was basically done.

Shipping it to the evaluator

This part deserves its own section because it cost us our entire first submission, and it is the kind of thing nobody writes down.

You submit to Flatland as a Docker image, so the evaluator pulls your image, runs your recorded-trajectory command, and scores it. The starter kit’s base image is flatland-baselines, which is 7.26 GB compressed because it carries conda, torch, ray, ffmpeg and the whole ML stack, none of which our planner needs, but we did not think about it, built on top of the base image, and submitted anyway.

The submission failed, and not with a crash, with this:

FAILURE  Elapsed time 1200.47s exceeded start time limit 1200s. Status: Pending

The evaluation pod gets 20 minutes to reach Running, and that includes pulling the image, and a 7.3 GB pull over the cluster’s registry link did not make it in time, so the pod sat Pending for exactly 1200 seconds and got killed. We had burned a submission (and we later confirmed the hard way that failed submissions still count against the daily cap of 2) on a problem that had nothing to do with our code.

The fix was a slim image, just python:3.12-slim plus flatland-rl==4.2.6 and nothing else, because the planner is pure Python and numpy, which is 825 MB instead of 7.3 GB and pulls in seconds. We kept the fat one around as Dockerfile.baselines just in case. There was also half an hour of fighting with GitHub Container Registry permissions, because giving the evaluator read access to a private package quietly turns off “inherit access from source repo”, which then takes away the CI’s write access so the build starts failing, and the two settings live in different panels so you need both. The lesson we took from this is simple: keep the image slim, and test the full pull-and-run path before you spend a submission on it.

The first real result

With the slim image (carrying a planner version we will get to, with overstay rights already folded in) we finally got a scored submission back, and here is where the whole thing stood after one day:

That is 20.74 out of 35, with 72.9% of trains delivered, and the public leaderboard leader at that moment was 12.49, so we were at rank 1 by a 66% margin, and our best planner version was still sitting unsubmitted because of the daily cap.

The per-level breakdown is the interesting part though, because it tells you exactly where the points are and turns “make it better” into an actual list:

• Levels 0 to 2 (no malfunctions): 12.60 out of 15, 100% delivered. This is working exactly as designed, and the gap from a perfect 15 is almost entirely our stop-skipping, which is the correct trade.
• Levels 3 to 5 (malfunctions): 7.19 out of 15, 56 to 78% delivered. This is the weak spot, because when trains break down our schedules cascade into jams and we lose a big chunk of the fleet, and this is where the points are.
• Level 6 (infrastructure disruptions): 0.94 out of 5, 15.6% delivered. This is almost a total loss, and it is a different kind of problem from the others.

So the clean levels were done, and the malfunction levels were where the competition was actually going to be won, and we had a guess about why they were failing.

The malfunction levels: two big fixes

To work on the malfunction problem locally we built some deliberately nasty stress environments, 62, 118 and up to 532 agents on the real map with random origin-destination pairs, mixed speeds and heavy malfunction rates, harsher than the real levels on purpose so that whatever survived them would be robust. The 118-agent storm was the one we used the most, and two fixes came out of it, and together they took that environment from 34% of trains delivered to 93%:

Fix 1: overstay rights

The first bug was subtle and nasty. When a train breaks down it sits in its cell for the malfunction duration, and every train queued behind it has to replan, but those followers had already reserved cells ahead of them, including the broken train’s own cell, for times in the near future. So when the broken train tried to replan its own escape, SIPP saw those followers’ reservations of its current cell as hard “you have to be gone by then” deadlines, and those were impossible to satisfy because the train was physically sitting there and could not leave until the malfunction ended, so SIPP returned no plan. Then the train went plan-less, which stubbed its cell, which made the followers replan, which re-reserved, and the whole queue behind one breakdown froze solid.

The fix we called overstay rights, which just says that a train that is physically in a cell may always stay in it, full stop. Its presence is the truth, and the other trains’ reservations of that cell are stale fiction that has to be re-timed around the truth, not treated as a constraint on the train that is actually there. So in the search, the start state of an on-map train gets a free pass to hold its own cell to the horizon, and when it replans we wake every train whose reservations conflicted with that cell so that they replan around reality:

# a replanning on-map train may keep its current cell indefinitely;
# wake every stale claimant so the queue learns the truth at once.
if agent.state not in OFF_MAP_STATES:
    for other in self.res.conflicting_owners(h, g, visits[:1]):
        self.dirty.add(other)

One detail that mattered here is that we wake all the conflicting claimants, not just the earliest one, because waking only the earliest lets the queue learn the truth one train at a time, which is too slow, and wedges still form while the news travels. If you wake all of them at once the whole queue re-times in a single step. On the worst storm environment this roughly doubled delivery on its own, from 34% up to 74.6%.

Fix 2: directional corridor locks

The second failure was the head-on deadlock from the swap section, but at scale. Watch two convoys come at the same long single-track artery from opposite ends. Each train on its own has a valid plan, but once the first train from each side commits into the artery they are on a collision course, and by the time the reservation conflict is visible they are already too deep to back out, because Flatland has no reverse. So they meet in the middle, cannot pass, cannot swap, and the artery is dead along with everything queued behind both of them.

The reservation table on its own does not stop this, because each train’s cells are free when it reserves them, so the conflict is a property of the whole corridor and not of any single cell. So we made the planner treat corridors as first-class objects. First we detect them, where a corridor is a maximal chain of plain cells, degree at most 2 with no routing choice, and on this map there are 471 of them, and the 18 long arteries (20 cells or more) are exactly where the face-offs happen. Here is what the solver sees:

The purple segments are the long arteries that get locked, and the teal ones are shorter corridors that resolve fine with normal cell reservations. Then the mechanism is that any plan that traverses a long artery locks it for that plan’s direction and time window, and an opposing plan that wants the same artery in the same window has to schedule after the lock releases, while same-direction platooning stays completely free, because trains chasing each other down the artery is fine, it is only the head-on case that has to be serialized:

In the search, before a train enters an artery it checks for an opposing lock over the window it wants to traverse, and if it finds one it waits at the mouth until it clears:

if est_transit:                       # entering a long artery
    sense_v = g.corridor_sense[v_node]
    for _ in range(8):
        lock_end = self.res.opposing_lock_until(seg_v, sense_v, tau, tau + est_transit, agent)
        if lock_end < 0:
            break                     # no opposing lock, go
        tau = lock_end                # wait for the artery to clear its direction

Tuning the length threshold was a small experiment on its own. Locking every corridor over-serializes the station throats, where short parallel tracks are meant to be used at the same time, and that cost us points, while locking only the long arteries hits just the handful of places where face-offs actually form. We swept the threshold, and 12 cells was too aggressive and 20 was the sweet spot, because it covers only the roughly 18 longest arteries. Here is a real face-off, an end-of-episode close-up of one contested artery on a 62-train scenario, with the lock off and then on:

Those red wedges are trains stopped nose to tail in a single-track artery, each one facing traffic it can neither pass nor reverse around. Without the lock both ends of the artery fill up and 16 trains are stranded, and with the lock the opposing traffic serializes at the mouths and the wedge shrinks. This particular artery still keeps a stubborn residual of 12 stuck trains even with the lock on, and every quick fix we threw at it (escape moves, locking the artery stubs) tested worse, so we left it, because it is worth about 0.1 normalized on a synthetic environment that may not even look like the real levels. But on the 118-train storm, where these face-offs are the main failure mode, corridor locks took delivery from 74.6% to 93.2%, which is 88 to 110 of the 118 trains home. Stacked on top of overstay rights, the two fixes together turned a 34%-delivery disaster into a 93%-delivery success.

Here is the shipped planner actually dispatching a 62-train scenario, colored by state, moving in teal and stopped in red, with most of the fleet flowing at once:

Across the full local benchmark suite the final configuration looks like this:

The 532-agent number is worth sitting with for a second, because the largest competition scenarios put 532 trains on this map, and the planner handles them at 97 to 100% delivery in about four minutes of wall-clock, well inside the 30-minute budget, so prioritized SIPP scales.

Things we tried that did not work

At least half of the work was ideas that sounded good and tested worse, and we measured every one of them with one-flag-at-a-time ablations instead of trusting our gut, and that paid off again and again, because a lot of the “obviously helpful” mechanisms were actively hurting. Roughly in the order we killed them:

• Headway buffers. The instinct from MAPF is to keep a one-step gap between trains, but in this reward regime a headway of 1 means every train that catches up to another has to stop to keep the gap, and every stop is -250 x v, so headway just does not fit the braking economics. We run headway = 0, so trains chain nose to tail, which is free, and only stop when they are actually blocked.

• Cascade replanning. When a train replans and its new path conflicts with others, the tempting move is to immediately replan all of them too, but it just churns, because the same conflicts re-form, planning time blows up, and the score does not move. We only wake the specific claimants of a train’s own current cell, which is the overstay case, and nothing more.

• Relaxed fallback planning. When the table was too crowded to find a conflict-free plan, we tried letting the train plan against only the static obstacles (stopped and broken trains) and sorting the rest out during execution, and this was a disaster, because trains committed to paths straight through moving traffic and produced exactly the head-on deadlocks we were trying to avoid. Off.

• Shift-repair and eager-repair. When a train is only delayed, why re-search from scratch, just push its remaining schedule later by the delay. We measured it, and it was -0.55 and -0.57 on the curriculum, because shifting keeps a route that a full replan would have routed around the new obstacle for free, so the full SIPP re-search is just better and both repairs are off.

• Escape moves. For a hopelessly stuck train, step it out of the way to break the jam. Net negative, because the escaping train just brakes (another -250) and parks somewhere that blocks a different route. Off.

• Serving intermediate stops. We tried turning stop-serving back on to collect the rebate, but it destabilizes dense scenarios, because halting at a platform is exactly what triggers the departure-delay malfunctions at levels 4 and 5, and the chaos that follows costs far more than the -50 rebates give back. The whole point was to skip, so leaving it off was right.

• Stub corridor locks. We tried having plan-less stubbed trains also lock their corridor, and it measured worse both ways, so locks stay traversal-only.

The lesson here is the same one from every optimization project, which is that the reward function knows more than you do and your intuition was built on a different reward function, so you measure everything instead of trusting it.

Repository layout

• submission/planner/ — the dispatcher. netgraph.py builds the (cell, direction) graph and detects the single-track corridors; sipp.py is Safe-Interval Path Planning over the shared reservation table, including overstay rights and the directional corridor locks; runtime.py is the plan-and-execute loop with priority planning, malfunction replanning, and the safety layers.
• submission/my_policy.py, submission/my_observation_builder.py — the policy and env-passing observation builder the evaluator calls.
• dev/ — the local evaluation harness (eval_lib.py, evaluate.py) that mirrors the server-side scoring, plus the stress environments and per-version result CSVs.

Reproducing locally

pip install flatland-rl==4.2.6

# score the SIPP planner on the example curriculum, with the official
# normalization and a per-penalty breakdown (mirrors the server-side eval)
python -m dev.evaluate --glob 'reinforcement_learning/curriculum/curriculum/example_curriculum/*.pkl'

Behaviour is controlled by environment-variable knobs, all defaulting to the measured-best configuration, for example PLANNER_LOCK_MIN_LEN=20 for the corridor-lock length gate and PLANNER_SERVE_STOPS=0 to keep stop-skipping on. See dev/SUBMISSION_NOTES.md for the full list and the tested-and-rejected ablations.

References:

1. ECML 2026 Flatland Real-World Baselines Challenge
2. Meyer et al., Real-World Baselines Challenge: Dynamic Train Rescheduling under Stochastic Perturbations, ECML PKDD 2026 Discovery Challenge
3. Flatland environment and evaluation code (flatland-rl)
4. Phillips and Likhachev, SIPP: Safe Interval Path Planning for Dynamic Environments, ICRA 2011
5. Silver, Cooperative Pathfinding (prioritized planning with reservation tables), AIIDE 2005

Table of Contents:

• V1: Just Sort The Thing
• V2: Let The Recursion Fan Out
• V3: Cutting The Branch (And Breaking Everything)
• V4: Going Wide With AVX-512

V1: Just Sort The Thing

Standard top down merge sort. Split the array in half, recurse on each half, merge the two sorted halves back together. The only real decision here is when to stop recursing, past a certain point the array is small enough that the overhead of splitting it further is not worth it so we just hand it to std::sort and let that handle the base case

typedef unsigned long long data_t;

void merge(
    data_t *left, int nl,
    data_t *right, int nr,
    data_t *out
) {
    int i=0, j=0, k=0;
    while(i < nl && j < nr) {
        if(left[i] < right[j]) out[k++] = left[i++];
        else out[k++] = right[j++];
    }
    if (i < nl) std::memcpy(out + k, left + i, (nl - i) * sizeof(data_t));
    else if (j < nr) std::memcpy(out + k, right + j, (nr - j) * sizeof(data_t));
}

void mergesort(data_t *src, int n, data_t *dst, int level) {
    if(n < 2) return;

    if(level <= MAX_LEVEL) {
        int mid = n / 2;
        mergesort(dst, mid, src, level + 1);
        mergesort(dst + mid, n - mid, src + mid, level + 1);
        merge(src, mid, src + mid, n - mid, dst);
    } else {
        std::sort(src, src + n);
        memcpy(dst, src, n * sizeof(data_t));
    }
}

void psort(int n, data_t *data) {
    data_t *buffer = (data_t *)aligned_alloc(64, ((n + 7) / 8) * 8 * sizeof(data_t));
    memcpy(buffer, data, n * sizeof(data_t));
    mergesort(buffer, n, data, 0);
    free(buffer);
}

Benchmarked this on my laptop (Apple M5, 10 cores, 4 performance + 6 efficiency, though this version doesn’t touch the extra cores at all yet) with random uint64 arrays, best of 3 runs each:

n                    time (ms)
1,000,000              23.60
10,000,000            241.84
100,000,000          2585.10

Roughly linear with a bit of superlinear creep as n grows, which tracks, merge sort is O(n log n) and the log n factor starts to bite once n crosses into the hundreds of millions

V2: Let The Recursion Fan Out

The two recursive calls in mergesort don’t depend on each other at all, left half and right half can happen at the same time, so this is the obvious place to throw threads at it. Wrapped both calls in #pragma omp task inside a taskgroup so the parent waits for both children before it merges. The one thing that actually matters here is capping how deep this fan-out goes, spawning a task for a sub array of like 8 elements is pure overhead, so MAX_LEVEL is picked so the recursion produces roughly 2 leaf tasks per thread and then falls back to plain std::sort underneath that

void mergesort(data_t *src, int n, data_t *dst, int level, const int MAX_LEVEL) {
    if(n < 2) return;

    if(level <= MAX_LEVEL) {
        int mid = n / 2;
        #pragma omp taskgroup
        {
            #pragma omp task
            mergesort(dst, mid, src, level + 1, MAX_LEVEL);
            #pragma omp task
            mergesort(dst + mid, n - mid, src + mid, level + 1, MAX_LEVEL);
        }
        merge(src, mid, src + mid, n - mid, dst);
    } else {
        std::sort(src, src + n);
        memcpy(dst, src, n * sizeof(data_t));
    }
}

void psort(int n, data_t *data) {
    data_t *buffer = (data_t *)aligned_alloc(64, ((n + 7) / 8) * 8 * sizeof(data_t));
    memcpy(buffer, data, n * sizeof(data_t));

    #pragma omp parallel
    #pragma omp single
    {
        int target_leaves = omp_get_max_threads() * 2;
        int MAX_LEVEL = (int)std::ceil(std::log2((double)target_leaves));
        mergesort(buffer, n, data, 0, MAX_LEVEL);
    }
    free(buffer);
}

n                    V1 (ms)     V2 (ms)     speedup
1,000,000              23.60       11.47       2.06x
10,000,000            241.84       82.52       2.93x
100,000,000          2585.10      760.88       3.40x

10 cores and best case is a 3.4x, not exactly the speedup you’d hope for. Three things eating into it: the M5 is 4 performance cores and 6 efficiency cores, so it’s not really 10 equal cores, the top level merge (combining the final two n/2 halves back into one) is always single threaded no matter how many cores you have which caps the whole thing by Amdahl’s law, and the smaller leaf tasks near the bottom of the recursion pay task creation overhead that doesn’t show up when you just eyeball the algorithm on paper. Nice to actually see the theory show up as real numbers instead of just trusting it

V3: Cutting The Branch (And Breaking Everything)

At this point the bottleneck inside each thread is just the merge loop, one comparison per element, left[i] < right[j]. On random data this branch is close to a coin flip so the CPU’s branch predictor is wrong close to half the time, which stalls the pipeline constantly. The idea was to process the compare loop in fixed size chunks of 32 so the loop overhead amortizes and the compiler has a better shot at pipelining the repeated branch pattern instead of treating every single comparison as a one-off

constexpr int BLOCK_SIZE = 32;

inline void merge(
    data_t *left, int nl,
    data_t *right, int nr,
    data_t *tmp, data_t *data
) {
    int i = 0, j = 0, k = 0;
    while (i < nl - BLOCK_SIZE && j < nr - BLOCK_SIZE) {
        for(int t=0; t<BLOCK_SIZE; t++) {
            if (left[i] <= right[j]) tmp[k++] = left[i++];
            else tmp[k++] = right[j++];
        }
    }
    while (i < nl) tmp[k++] = left[i++];
    while (j < nr) tmp[k++] = right[j++];
    std::memcpy(data, tmp, (nl + nr) * sizeof(data_t));
}

To actually find out how much of the merge cost was the branch itself versus just moving memory around, I wrote a second version of merge that skips the comparison entirely and always pulls from the left, purely to measure the floor:

inline void merge2(
    data_t *left, int nl,
    data_t *right, int nr,
    data_t *tmp, data_t *data
) {
    int i = 0, j = 0, k = 0;
    while (i < nl - BLOCK_SIZE && j < nr - BLOCK_SIZE) {
        for(int t=0; t<BLOCK_SIZE; t++) { k++; i++; }
    }
    while (i < nl) tmp[k++] = left[i++];
    while (j < nr) tmp[k++] = right[j++];
    std::memcpy(data, tmp, (nl + nr) * sizeof(data_t));
}

Obviously this doesn’t sort anything correctly since it’s not comparing at all, that’s the whole point, it’s just there to answer “how fast would this be with the branch removed.” Plugged it into the same parallel task structure as V2 and at n=10,000,000 the real branchy merge from V2 runs in 82.52ms while this fake branchless one runs in 48.90ms. So somewhere around 40% of the merge time was the mispredicted branch itself, the rest is just memory traffic you can’t get rid of

Then I went back and swapped the real chunked comparison back in, expecting something between those two numbers. Got 110.28ms, slower than V2, and std::is_sorted came back false again. So there’s a real bug in here somewhere, probably in how the blocking interacts with the parallel task recursion sharing the same tmp buffer across levels, and I never actually tracked it down. This version still sits broken in the so4 folder. V2 stayed the real answer

Worth saying plainly: I almost reported the merge2 number as a real speedup before running is_sorted on it. A fast wrong answer is worse than a slow right one, and it’s very easy to only check the timer

V4: Going Wide With AVX-512

The scalar merge loop only ever looks at one element per side per comparison. AVX-512 lets you compare 16 lanes at once, so instead of merging one element at a time you can merge whole tiles at a time

The approach has two phases. First, sort small 16 element tiles entirely inside a register using a fixed sequence of compare-and-swap stages, a bitonic sorting network, so a tile never touches memory until it’s fully sorted:

static inline void sort16_inregister(__m512i *v) {
    __m512i x = *v, y, min, max;
    y   = _mm512_shuffle_epi32(x, _MM_PERM_CDAB);
    min = _mm512_min_epi32(x, y);
    max = _mm512_max_epi32(x, y);
    x   = _mm512_blend_epi32(min, max, 0xAA);

    y   = _mm512_shuffle_epi32(x, _MM_PERM_BADC);
    min = _mm512_min_epi32(x, y);
    max = _mm512_max_epi32(x, y);
    x   = _mm512_blend_epi32(min, max, 0xCC);

    y   = _mm512_shuffle_epi32(x, _MM_PERM_4321);
    min = _mm512_min_epi32(x, y);
    max = _mm512_max_epi32(x, y);
    x   = _mm512_blend_epi32(min, max, 0xF0);

    y   = _mm512_shuffle_epi32(x, _MM_PERM_CDAB);
    min = _mm512_min_epi32(x, y);
    max = _mm512_max_epi32(x, y);
    *v  = _mm512_blend_epi32(min, max, 0xAA);
}

Second, merge sorted runs bottom up, doubling the run length each pass like a normal iterative merge sort, but merging 16 elements from each side at a time instead of one. Compare both 16-lane chunks to get a mask of which lane came from which side, popcount that mask to know exactly how many elements the left side contributed this round, and use a masked compress-store to pack just those lanes contiguously into the output, all in one instruction instead of 16 branchy comparisons:

static void merge_avx512(const int *A, const int *B, int *OUT, size_t N) {
    size_t i = 0, j = 0, k = 0;
    while (i + 16 <= N && j + 16 <= N) {
        __m512i bufA = _mm512_loadu_si512((__m512i*)(A + i));
        __m512i bufB = _mm512_loadu_si512((__m512i*)(B + j));
        __mmask16 m = _mm512_cmple_epi32_mask(bufA, bufB);
        size_t na = _popcnt32(m);
        _mm512_mask_compressstoreu_epi32(OUT + k,      m,  bufA);
        _mm512_mask_compressstoreu_epi32(OUT + k + na, ~m, bufB);
        i += na; j += 16 - na; k += 16;
    }
    while (i < N && j < N) OUT[k++] = (A[i] < B[j] ? A[i++] : B[j++]);
    while (i < N) OUT[k++] = A[i++];
    while (j < N) OUT[k++] = B[j++];
}

void merge_sort_avx512_omp(int *data, size_t N) {
    const size_t TILE = 16;
    int *buffer = (int*)aligned_alloc(64, N * sizeof(int));

    #pragma omp parallel for schedule(static)
    for (size_t i = 0; i + TILE <= N; i += TILE) {
        __m512i v = _mm512_loadu_si512((__m512i*)(data + i));
        sort16_inregister(&v);
        _mm512_storeu_si512((__m512i*)(data + i), v);
    }

    for (size_t width = TILE; width < N; width <<= 1) {
        #pragma omp parallel for schedule(dynamic)
        for (size_t start = 0; start < N; start += 2 * width) {
            size_t mid = start + width;
            if (mid >= N) continue;
            size_t end = std::min(mid + width, N);
            merge_avx512(data + start, data + mid, buffer + start, mid - start);
            memcpy(data + start, buffer + start, (end - start) * sizeof(int));
        }
    }
    free(buffer);
}

As a corollary there is also no point is choosing a very ambigious goal for yourself or choosing to work on a very very ambitious problem but then deciding to go via shortcuts to achieve it. There is also no point in doing hard work and sweating over problems that are too small to waste your time on

If you have decided to work on a very ambigious goals for your self and also decided to walk on that path, you need to be comfortable with failing, have pragmatism even in the worst of times, relentless pursuit to pioneer and the exceptional ability to never give up. If you are afraid of taking decisions or saying stuff because you think it will make you look boring, cringe, you are not going to make it and it’s a signal that you are again diverting from the path that you decided to once follow

This is a good enough proxy or good enough signal to measure if the things that I have decided to work on or problems which I am solving, are they worth solving and will they eventually lead to something important and will they make my life and the life of everyone around me much better

• Don’t Die: in the sense that don’t ever go to the point of no return, where you can never get back up. Be obsessed with the death boundaries and downside scenarios and always prioritize survival
• Never Quit: Things take time, some people die from an unlucky accident on a random day, some people make the fortune of their life in a day but these are rather fleeting events. Sharp rises are always met with sharp declines. If you set your mind to something then why quit what changed? Darshan if you followed this framework you are not dead then keep moving you are still in the game. “Too early” happens only when you quit. When you feel down or feel quitting, score yourself on a different scoreboard and start compounding the results. Some of the best things started slow (pytorch, transformer, nvidia) and then compounded to heaven. Have patience. slowness is needed when you are finding out your goal on a map
• Lock In: Be obsessed over the thing that you are working on, Always be in the game observe the game and their players, how they are playing the game. There are many players playing the same game which you are. If you are not spending 100% of your focus on the game, you are already out of the game. There will be many things that prevent you to get lock in 100%, think about ways to delete, defer or delegate these tasks
• Do Hard Things: Doing hard things is weird but that is okay because winning is different and makes you weird and means that you are different from others. You will look like a noob and a fool frankly when you do hard things. They will underestimate you but you are compounding. You will create a massive vertical between you and the next participant of the game before they know what hit them. People by nature take the path of least resistance, don’t be like them. It’s very easy to build the surface thing but the right (hard) infrastructure, mental model or technology compounds. Doing hard things allows you to go deeper into things and gain advantage over others
• Play by the Rules: This is one thing that the universe does not reward. There are people who don’t play by the rules but they are exceptions or their time will come. Don’t be tempted to do the same. Stay focused. Playing by the rules is hard. But it is the hard things in life that force you to innovate your way out of problems. These innovations compound in weird and interesting patterns resulting in long term moats. There is an irish revolutionary song that encapsulates “The higher you build your barricades the stronger we become”
• Don’t chase after thin desires: thin desires are an everlasting hunger for things that we can’t articulate why we want them, and cannot actually attain them, from a source that has no interest in providing them. Chase thick desires instead, they transform/shape/mold you in the act of pursuing them. Thick desires are effortful to acquire and are impossible to gratify. Thick desires also make you antifragile. It’s completely fine to have desires but make sure they are worth pursuing and they will shape you in the process of pursuing them
• Put Something Back: Putting something back seems neat, but it also helps you in the long term. Karms seems real. When you put something back into the pool, your reputation grows, people will be familiar with your work and they will want to be around you and work for you or with you. Winning can be compounded by giving back. Win and give to win more later
• Don’t become the machine: Everyone seems to praise the hustle culture these days, you have to work harder than that, you have to push yourself everyday, you have to wake up at 5am and yade yada. You don’t have to become a machine to succeed. Just do the work at your body’s natural pace. Your body will tell you when it’s enough and what it’s limits are. Efficiency is a measure for the machines, please don’t eval yourself like a machine. Aim to be nimble, adapt quickly to uncertainties, be anti fragile
• Update your mental models: When you know so little, you don’t know how much there is to know and things feel trivial and everything you think is clear. You are also more certain about it. Infact certainty peaks before competence in the dunning kruger curve. The less you know the more you seem to be confident about it. It’s far riskier to be certainly incorrect most of the time than be uncertainly correct all the time.
• Cultivate Confidence: Invest in things that challenge your confidence and let your anxiety go away. Being shy is not an excuse for being rude. Confidence in other player’s cooperation also influences our own actions. Having negative interactions can lead to a downward spiral going all the way down and taking you with it. Have positive interactions with people and be around people with very high urgency and agency. The positive spiral can take you to the stairway to heaven
• You have to be strong: This shit ain’t for the weak. They will eat you, they will hunt you, they will abuse you, they will laugh at you, they want you to be down, every time you spent crying, spiraling downward they will win. The trick is to adopt the stoic principles of focusing no what you can control and ignore what you can’t. Your can’t control when a tragedy is going to happen but you can control how you respond and what you learn from it
• Invest in People: I believed that your work speaks for itself, no it doesn’t it just sits silently in your repository. Your manager mentions you in a conversation or a meeting. A peer who you spent 5 mins chatting with you gets impressed and recommends you for a short project. Decisions get made in meetings you are not invited, using summaries you didn’t wrote by people who have 5 mins and 12 different tasks to work on. If no one can articulate your impact when you are not in the room then your impact is optional. Make the value chain legible to everyone including yourself

So after this I just went away with this setup. The first couple of days were not bad I mean my legs were already in pretty good shape as I have a regular running habit. I wasn’t feeling fatigued or lethargic at all, I would say I was more aware of my own physical body or how much energy/force it has to exert while doing a task which I used to do just sitting. I was just slightly distracted by this fact but not distracted enough to not be able to do any work at all. After the first week it became natural to do long hours of work in this new setup of mine.

I have observed that the speed at which I am approaching and getting things done has slightly increased. I have also become more proactive and more communicative. I now love standing while working. I have now kept two 10L water jugs alongside my desk and I have been experimenting with it doing bicep curls/hammer curls/triceps in every 30mins interval, I have also experimented with adding pushups in this combo, it has been working out fantastic for me. I think this is clear connection between productivity and slight pressure to body to keep it in order, every time we lift weights or make our muscles work under pressure we are unconsciously reminding our mind that we are control of our physical selves and our actions does have meaningful impact in the world that we live in

I remain energetic even after long hours of deep work, I don’t get tired after a long day which I usually felt previously. I don’t think weather the gamified setup of exercise + work or the standing desk is the causation of this, but I can confirm there is definitely a correlation and it has been working out fanstastic for me

I gave a couple of interviews in the last months and I cleared a majority of them while standing. I now speak more energetically and aggressively while I am standing. I have noticed that when I don’t get exercise or my body does not undergo any form of activity I simply cannot concentrate and if I am in chair then end up spending my day chaining positions

what is a gaussian splat?

a 3D Gaussian splat is an oriented ellipsoid in space that carries some color and opacity. you can think of it as a fuzzy colored blob. a scene is made of hundreds of thousands of these blobs, and when you look at them from a particular viewpoint, they overlap and blend to form the final image

We represent each gaussian with these attributes:

pub struct Splat {
    pub pos: Vec3,      // center position in world space
    pub scale: Vec3,    // size along each local axis
    pub rot: Quat,      // orientation as a unit quaternion
    pub color: Vec3,    // RGB color (already decoded from spherical harmonics)
    pub opacity: f32,   // how opaque this blob is, in [0, 1]
}

scales are stored in log space, opacities as logits, colors as spherical harmonics coefficients and quaternions are normzlied to unit length to ensure the values lie within their respective range

spherical harmonic (SH) coefficients are just a frequency-domain representation of a color function defined over the unit sphere, now why spherical harmonics? because in the real world, the color of a surface depends on the viewing direction. SH coefficients encode this view-dependent appearance compactly.

SH functions are organized in bands (like octaves in music), as you go higher up in the bands you have more coefficients and thus they capture more finer details, the INRIA 3DGS format stores up to band 3 (48 coefficients per splat for RGB)

To decode bash 0, the band-0 SH basis function is $Y_0^0 = \frac{1}{2\sqrt{\pi}} \approx 0.282$. the conversion from SH coefficient to RGB is:

where $C_0 = Y_0^0$ and $f_{dc}$ is the 3-component DC coefficient from the file.

pub const SH_C0: f32 = 0.28209479177387814;

pub fn sh_band0_to_rgb(f_dc: Vec3) -> Vec3 {
    (Vec3::splat(0.5) + SH_C0 * f_dc).clamp(Vec3::ZERO, Vec3::ONE)
}

the forward pass pipeline

the forward pass turns a list of 3D Gaussians + a camera into a 2D image. here is an overview of the rendering pipeline:

Step 1: Projecting Splats

1.1: building the 3D covariance matrix

for each splat given the raw (scale, rotation) pairs we need to construct a 3D covariance matrix $\Sigma$ that describes the shape and orientation of the Gaussian in world space. the formula is:

where R is the 3×3 rotation matrix from the quaternion, and S is a diagonal matrix of scales. if we let M = R·S, this simplifies to:

let r_mat = Mat3::from_quat(s.rot);
let s_mat = Mat3::from_diagonal(s.scale);
let m = r_mat * s_mat;
let cov3d = m * m.transpose();

Note: why dowe decompose the covariance this way?

Covariance matrices have physical meaning only when they are positive semi-definite. gradient descent cannot easily be constrained to produce valid matrices, by expressing the covariance as $M \cdot M^T$, it is guaranteed to be positive semi-definite, a matrix of the form $A^T A$ always is. this is a reparametrization trick: we optimize scale and rotation separately, which are unconstrained, and the covariance we derive from them is always valid

what does this matrix actually look like? for a splat with scale = (0.1, 0.05, 0.02) and identity rotation:

with identity rotation, it is just the squared scales on the diagonal, an axis-aligned ellipsoid

1.2: transforming into view space

the 3D covariance we just computed lives in world space. to project it onto the camera’s image plane, we first need to rotate it into view space, the coordinate system where the camera is at the origin, looking down −z

for the splat center, this is just a matrix-vector multiply with the 4×4 view matrix:

let p_view4 = view * Vec4::new(s.pos.x, s.pos.y, s.pos.z, 1.0);
let p_view = Vec3::new(p_view4.x, p_view4.y, p_view4.z);
if p_view.z > -znear || p_view.z < -zfar {
    return None;
}
let zc = -p_view.z;

note zc = -p_view.z. our view space is right-handed with the camera looking down −z, so points in front of the camera have negative z. we use zc (positive in front) as the depth for sorting and projection.

for the covariance, we rotate it by the 3×3 part of the view matrix W:

let w_mat = Mat3::from_mat4(view);
let w_mat_t = w_mat.transpose();

let cov3d_view = w_mat * cov3d * w_mat_t;

this is just the standard basis-change formula for a covariance matrix. the shape of the ellipsoid does not change due to this infact we are only re-expressing it in the camera’s coordinate system.

1.3: projecting to 2D

now we have a 3D Gaussian in view space and we need to project it onto the 2D image plane. the projection is perspective, which means a 3D Gaussian does not project to an exact 2D Gaussian because perspective is a nonlinear transform. but we can locally linearize it using the Jacobian of the projection function, and the result is close enough.

the projection function maps a 3D point (x, y, z) in view space to pixel coordinates (u, v):

\(u = f_x \cdot \frac{x}{z_c} + c_x\) \(v = f_y \cdot \frac{y}{z_c} + c_y\)

where $f_x, f_y$ are the focal lengths and $c_x, c_y$ are the principal point (image center). for simplicity of calcuations we can also assume, $f_x = f_y$

the Jacobian J of this projection evaluated at the splat center is:

the structure of this matrix is very sparse, only 4 of the 9 entries are nonzero.we can just do the full $JC J^T$ with two 3×3 matrix multiplies (~54 scalar multiplies). but because we only need the top-left $2\times 2$ of $J$ Cov3D_view $J^T$, since the third row of $J$ is all zeros. and the first two rows of J each have only two nonzero entries. so instead of two full matrix multiplies, we can compute the 2D covariance with ~20 scalar multiplies by expanding the product by hand:

let c = &cov3d_view;
let inv_zc = 1.0 / zc;
let inv_zc2 = inv_zc * inv_zc;

let j00 = fx * inv_zc;
let j02 = fx * xv * inv_zc2;
let j11 = fy * inv_zc;
let j12 = fy * yv * inv_zc2;

// Row 0 of J * C: [j00*c00 + j02*c20, j00*c01 + j02*c21, j00*c02 + j02*c22]
let t0x = j00 * c.x_axis.x + j02 * c.z_axis.x;
let t0y = j00 * c.y_axis.x + j02 * c.z_axis.y;
let t0z = j00 * c.x_axis.z + j02 * c.z_axis.z;

// Row 1 of J * C: [j11*c10 + j12*c20, j11*c11 + j12*c21, j11*c12 + j12*c22]
let t1y = j11 * c.y_axis.y + j12 * c.z_axis.y;
let t1z = j11 * c.y_axis.z + j12 * c.z_axis.z;

// 2D cov = (J*C) * J^T, top-left 2x2:
let cov2d_00 = t0x * j00 + t0z * j02 + eps2d;
let cov2d_01 = t0y * j11 + t0z * j12;
let cov2d_11 = t1y * j11 + t1z * j12 + eps2d;

notice the eps2d on the diagonal entries. that is a small dilation (default 0.3) added for numerical stability, it just ensures the 2D covariance is strictly positive definite (not just semi-definite), which means it is always invertible.

Note: why the eps2d trick works

by construction, the 2D covariance $JCJ^T$ is only positive semi-definite ($A^T A$ form). but we need to invert it later (for evaluating the Gaussian at each pixel). a singular matrix is not invertible.

adding eps2d to the diagonal means adding $\lambda I$ to the matrix. for any vector x:

this is strictly positive for any nonzero x, which is the definition of positive definite, invertible, with all eigenvalues strictly positive

next we invert the $2\times 2$ covariance. for a $2\times 2$ matrix the inverse has a closed form:

let det = cov2d_00 * cov2d_11 - cov2d_01 * cov2d_01;
if det <= 0.0 {
    return None;
}

let inv_det = 1.0 / det;
let cov2d_inv = Mat2::from_cols(
    Vec2::new(cov2d_11 * inv_det, -cov2d_01 * inv_det),
    Vec2::new(-cov2d_01 * inv_det, cov2d_00 * inv_det),
);

and the screen position is a standard perspective divide:

let sx = fx * xv * inv_zc + cx;
let sy = fy * yv * inv_zc + cy;

at this point, for each splat we have: screen position (sx, sy), depth zc, and the inverse 2D covariance matrix cov2d_inv. this is everything we need to evaluate the Gaussian at any pixel on screen.

step 2: computing bounding boxes

We would now have to evalute the splat at every pixel in the screen because the 2D gaussian has infinite support, it never truly reaches zero, but we can compute a bounding box that encloses the region where the gaussian has any visible effect and only evalute pixels inside this bounding box

the original 3DGS code computes the two eigenvalues $\lambda_1$, $\lambda_2$ of the 2D covariance (the variances along the two principal axes of the ellipse), takes $r = 3\sqrt{max(\lambda_1, \lambda_2)}$ (the 3-sigma rule, covering 99.7% of the Gaussian), and uses a circle of that radius as the bounding box

this is simple but wasteful. when a Gaussian is elongated (one eigenvalue much larger than the other), the bounding circle will also include a lot of empty space

We can create a tighter bounding box by observing that:

1. the extent along each axis is $k\sqrt{\Sigma_{ii}}$ where $\Sigma_{ii}$ is the diagonal entry of the 2D covariance for that axis. for an elongated ellipse, the short-axis extent is much smaller than the long-axis extent, so the bounding box is tighter.

2. the classic $3\sigma$ rule is conservative. a faint splat (low opacity) does not need $3\sigma$ because its contribution drops below the visibility threshold much sooner. the cutoff distance $k$ can be computed when our splat’s opacity falls below certain threshold $\tau$

based on this each pixel gets

we want $\alpha \geq \tau$, which rearranges to:

for a near-opaque splat (opacity = 1), $k^2 = 2ln(255) = 11.1$, which is close to the $3\sigma$ value of 9. for a faint splat (opacity = 0.1), $k^2 = 2ln(255) = 11.1$, so the box and computation shrinks substantially.

if s.opacity <= alpha_threshold {
    return None;
}
let k2 = (2.0 * (s.opacity / alpha_threshold).ln()).min(max_k2);
if !(k2 > 0.0) {
    return None;
}

let rx_f = (k2 * cov2d_00).sqrt();
let ry_f = (k2 * cov2d_11).sqrt();
if !rx_f.is_finite() || !ry_f.is_finite() || rx_f < 1.0 || ry_f < 1.0 {
    return None;
}

let x0 = (sx - rx_f).floor() as i32;
let y0 = (sy - ry_f).floor() as i32;
let x1 = (sx + rx_f).ceil() as i32;
let y1 = (sy + ry_f).ceil() as i32;

// Clip to framebuffer.
let x0 = x0.max(0);
let y0 = y0.max(0);
let x1 = x1.min(w_i - 1);
let y1 = y1.min(h_i - 1);
if x0 > x1 || y0 > y1 {
    return None;
}

Also because each splat is independent of each other the projection computation is embarassingly parallel, we use rayon’s par_iter().filter_map() to project all splats across all cores:

pub fn project(
    splats: &[Splat],
    camera: &OrbitCamera,
    params: &RenderParams,
    pool: &Option<rayon::ThreadPool>,
) -> Vec<Projected> {
    // ... precompute view matrix, intrinsics, etc (once per frame) ...

    let do_project = || {
        splats
            .par_iter()
            .filter_map(|s| {
                // ... all the math above, returning Some(Projected) or None ...
            })
            .collect()
    };

    match pool.as_ref() {
        Some(p) => p.install(do_project),
        None => do_project(),
    }
}

we can now just pack everything into a Projected struct

pub struct Projected {
    pub screen: Vec2,       // pixel center (sx, sy)
    pub depth: f32,         // zc (positive in front)
    pub cov2d_inv: Mat2,    // inverse 2D covariance
    pub bbox: [i32; 4],     // inclusive: x0, y0, x1, y1
    pub color: Vec3,        // RGB
    pub opacity: f32,       // [0, 1]
}

step 3: depth sort

For alpha compositing we use painter’s algorithm in reverse. this means we need the projected splats sorted by depth before compositing. the order matters here, if a close splat occludes a far one, it must be composited first so it has more weight in the blending process. because all depths are positive (we culled anything behind the camera), so the bit pattern of an f32 preserves float ordering when reinterpreted as a u32. this lets us use depth.to_bits() as a sort key

for the actual sorting, we use a simple 2-pass 16-bit radix sort. this is faster than comparison-based sorting (like Rust’s sort_unstable_by_key) for our input sizes (~100k–200k elements) and it runs in O(n) time with small constants. 2 passes of 16 bits seemed to have worked better instead of 4 passes of 8 bits, fewer passes means fewer data traversals and also the 65536-entry histograms (256KB each) fit comfortably in my laptops L2 cache. at 200k splats this is consistently 2 times faster than Rust’s comparison sort.

pub fn sort_by_depth(projected: &mut [Projected], scratch: &mut ScratchBuffers) {
    let n = projected.len();
    if n <= 1 {
        return;
    }

    scratch.sort_aux.clear();
    scratch.sort_aux.reserve(n.saturating_sub(scratch.sort_aux.capacity()));
    unsafe { scratch.sort_aux.set_len(n); }
    let aux = &mut scratch.sort_aux;

    // Both histograms can be computed in a single pass over the keys.
    // Stack-allocated to avoid heap alloc.
    let mut counts_lo = [0u32; 65536];
    let mut counts_hi = [0u32; 65536];
    for p in projected.iter() {
        let k = p.depth.to_bits();
        counts_lo[(k & 0xFFFF) as usize] += 1;
        counts_hi[(k >> 16) as usize] += 1;
    }

    // Pass 1: sort by low 16 bits
    {
        let mut offsets = [0u32; 65536];
        let mut sum = 0u32;
        for i in 0..65536 {
            offsets[i] = sum;
            sum += counts_lo[i];
        }
        for p in projected.iter() {
            let bucket = (p.depth.to_bits() & 0xFFFF) as usize;
            let pos = offsets[bucket] as usize;
            offsets[bucket] += 1;
            aux[pos] = *p;
        }
    }

    // Pass 2: sort by high 16 bits
    {
        let mut offsets = [0u32; 65536];
        let mut sum = 0u32;
        for i in 0..65536 {
            offsets[i] = sum;
            sum += counts_hi[i];
        }
        for p in aux.iter() {
            let bucket = (p.depth.to_bits() >> 16) as usize;
            let pos = offsets[bucket] as usize;
            offsets[bucket] += 1;
            projected[pos] = *p;
        }
    }
}

pub fn sort_by_depth_parallel(
    projected: &mut [Projected],
    scratch: &mut ScratchBuffers,
    pool: &Option<rayon::ThreadPool>,
) {
    if projected.len() < 50_000 || pool.is_none() {
        sort_by_depth(projected, scratch);
        return;
    }
    pool.as_ref().unwrap().install(|| {
        projected.par_sort_unstable_by_key(|p| p.depth.to_bits());
    });
}

step 4: tile binning

we now have a depth-sorted list of projected splats. the naive approach to compositing would be: for each splat, iterate every pixel in its bounding box and accumulate color. this works but it is cache-hostile because different splats touch overlapping pixel regions in unpredictable order.

the original 3DGS implementation divides the image into tiles (16×16 pixel blocks) and builds an index that tells each tile exactly which splats overlap it. then each tile composites only its own splats, in order, touching only its own pixels

the binning uses a count-then-scatter two-pass approach. in pass 1 for each splat, count how many tiles its bbox touches then build a per-tile count array. in pass 2, prefix-sum the counts into offsets (so offsets[i] = start of tile i’s bucket). then scatter each splat’s index into its tile buckets using a cursor that advances. now because we iterate splats in depth-sorted order, each tile’s bucket ends up automatically depth-sorted so no additional per-tile sort needed

pub fn bin_splats(
    projected: &[Projected],
    width: u32,
    height: u32,
    bins: &mut TileBins,
) {
    let num_tiles_x = ((width as i32) + TILE_W - 1) / TILE_W;
    let num_tiles_y = ((height as i32) + TILE_H - 1) / TILE_H;
    let num_tiles = (num_tiles_x * num_tiles_y) as usize;

    // offsets is used first as a count array, then prefix-summed in place.
    bins.offsets.clear();
    bins.offsets.resize(num_tiles + 1, 0);

    // Pass 1: count tile touches per splat.
    for p in projected {
        let [x0, y0, x1, y1] = p.bbox;
        let tx0 = (x0 / TILE_W).max(0);
        let ty0 = (y0 / TILE_H).max(0);
        let tx1 = (x1 / TILE_W).min(num_tiles_x - 1);
        let ty1 = (y1 / TILE_H).min(num_tiles_y - 1);
        if tx0 > tx1 || ty0 > ty1 {
            continue;
        }
        for ty in ty0..=ty1 {
            let row = (ty * num_tiles_x) as usize;
            for tx in tx0..=tx1 {
                bins.offsets[row + tx as usize + 1] += 1;
            }
        }
    }

    // Prefix sum: offsets[i] = start of bucket i.
    for i in 1..=num_tiles {
        bins.offsets[i] += bins.offsets[i - 1];
    }
    let total = bins.offsets[num_tiles] as usize;

    bins.splat_indices.clear();
    bins.splat_indices.resize(total, 0);

    // Cursor starts at each bucket's begin and advances as we scatter.
    bins.cursor.clear();
    bins.cursor.extend_from_slice(&bins.offsets[..num_tiles]);

    // Pass 2: scatter splat indices into their tile buckets.
    for (idx, p) in projected.iter().enumerate() {
        let [x0, y0, x1, y1] = p.bbox;
        let tx0 = (x0 / TILE_W).max(0);
        let ty0 = (y0 / TILE_H).max(0);
        let tx1 = (x1 / TILE_W).min(num_tiles_x - 1);
        let ty1 = (y1 / TILE_H).min(num_tiles_y - 1);
        if tx0 > tx1 || ty0 > ty1 {
            continue;
        }
        for ty in ty0..=ty1 {
            let row = (ty * num_tiles_x) as usize;
            for tx in tx0..=tx1 {
                let tile = row + tx as usize;
                let pos = bins.cursor[tile] as usize;
                bins.splat_indices[pos] = idx as u32;
                bins.cursor[tile] += 1;
            }
        }
    }
}

let me show what the data structure looks like for a small example. say we have 4 splats (A, B, C, D) then after depth sorting we will get something like this

step 5: alpha compositing

now for each tile, we iterate its splats front-to-back, and for each pixel in the splat’s bounding box (clipped to the tile), we evaluate the 2D Gaussian and blend the color.

for a pixel at position $(px, py)$ and a splat centered at $(sx, sy)$, the displacement is $d = (px - sx, py - sy)$. the Gaussian exponent is:

expanding for a 2×2 symmetric inverse covariance with entries $(a, b; b, d)$:

the alpha for this splat at this pixel is:

we now composite front-to-back. the framebuffer stores (rgb_accum, alpha_accum) per pixel, both starting at zero. for each splat, the contribution is:

\(T = 1 - \alpha_{accum}\) \(\text{contrib} = T \cdot \alpha\) \(\text{rgb}_{accum} \mathrel{+}= \text{contrib} \cdot \text{color}\) \(\alpha_{accum} \mathrel{+}= \text{contrib}\)

here $T$ is the transmittance i.e how much light can still pass through. once alpha_accum reaches the saturation threshold (0.999), the pixel is considered opaque and we skip further splats on it.

the inner pixel loop is the hottest code in the entire rasterizer. to make it as fast as possible, we hoist row-constant terms outside the inner (column) loop. for a fixed row $p_y$, $d_y = p_y - s_y$ is constant, so we precompute everything else and then the inner loop becomes a simple quadratic in dx with all coefficients precomputed

fn composite_splat(p: &Projected, fb: &mut [(Vec3, f32)], w: usize, params: &RenderParams) {
    let [x0, y0, x1, y1] = p.bbox;

    // Extract the inverse covariance matrix elements.
    let a = p.cov2d_inv.x_axis.x; // (0,0)
    let b = p.cov2d_inv.x_axis.y; // (0,1) = (1,0) since symmetric
    let d = p.cov2d_inv.y_axis.y; // (1,1)

    // Coefficients for the decomposed quadratic.
    let dx_coeff = -0.5 * a;
    let dy_coeff = -0.5 * d;
    let cross_coeff = -b; // -0.5 * 2 * b

    let saturation = params.saturation;
    let alpha_threshold = params.alpha_threshold;
    let opacity = p.opacity;
    let color = p.color;
    let sx = p.screen.x;
    let sy = p.screen.y;

    for py in y0..=y1 {
        let dy = py as f32 - sy;
        let row_base = dy_coeff * dy * dy;
        let row_slope = cross_coeff * dy;
        let row_offset = py as usize * w;

        for px in x0..=x1 {
            let idx = row_offset + px as usize;
            let cell = &mut fb[idx];
            if cell.1 >= saturation {
                continue;
            }
            let dx = px as f32 - sx;
            let power = dx_coeff * dx * dx + row_slope * dx + row_base;
            if power > 0.0 {
                continue;
            }
            let alpha = (opacity * fast_exp(power)).min(0.999);
            if alpha < alpha_threshold {
                continue;
            }
            let t = 1.0 - cell.1;
            let contrib = t * alpha;
            cell.0 += contrib * color;
            cell.1 += contrib;
        }
    }
}

tiled parallel compositing

now because each tile owns a disjoint pixel sets so we can composite all tiles in parallel with zero synchronization, no atomics and no locks, this becomes very effective on CPU

we use rayon’s par_iter over tile indices, and raw-pointer writes to give each tile direct access to its pixel rectangle without violating Rust’s &mut aliasing rules:

pub fn composite_tiled(
    projected: &[Projected],
    bins: &TileBins,
    fb: &mut [(Vec3, f32)],
    width: u32,
    height: u32,
    params: &RenderParams,
    pool: &Option<rayon::ThreadPool>,
) {
    let w = width as usize;
    let h_i = height as i32;
    let w_i = width as i32;
    let num_tiles_x = bins.num_tiles_x;
    let num_tiles = bins.num_tiles();
    let fb_ptr = FbPtr(fb.as_mut_ptr());

    let do_composite = || {
        (0..num_tiles).into_par_iter().for_each(move |tile_idx| {
            let fbp = fb_ptr;
            let tile_x = (tile_idx as i32) % num_tiles_x;
            let tile_y = (tile_idx as i32) / num_tiles_x;
            let px0 = tile_x * TILE_W;
            let py0 = tile_y * TILE_H;
            let px1 = (px0 + TILE_W - 1).min(w_i - 1);
            let py1 = (py0 + TILE_H - 1).min(h_i - 1);
            if px0 > px1 || py0 > py1 {
                return;
            }

            let start = bins.offsets[tile_idx] as usize;
            let end = bins.offsets[tile_idx + 1] as usize;
            if start == end {
                return;
            }
            let splat_ids = &bins.splat_indices[start..end];

            for &sid in splat_ids {
                let p = unsafe { projected.get_unchecked(sid as usize) };
                let [bx0, by0, bx1, by1] = p.bbox;
                let x0 = bx0.max(px0);
                let y0 = by0.max(py0);
                let x1 = bx1.min(px1);
                let y1 = by1.min(py1);
                if x0 > x1 || y0 > y1 {
                    continue;
                }
                unsafe {
                    composite_splat_region(p, fbp.0, w, x0, y0, x1, y1, params);
                }
            }
        });
    };

    match pool.as_ref() {
        Some(p) => p.install(do_composite),
        None => do_composite(),
    }
}

Note: the FbPtr trick

Rust’s borrow checker does not let multiple threads hold &mut references to the same buffer. but we know that each tile writes to a disjoint pixel rectangle the geometry guarantees it. so we wrap the raw pointer in a Send + Sync newtype and give each tile direct access:

struct FbPtr(*mut (Vec3, f32));
unsafe impl Send for FbPtr {}
unsafe impl Sync for FbPtr {}

this is the one place in the rasterizer where we need unsafe. the safety argument is spatial here and tile (tx, ty) only writes to pixels [tx*16..(tx+1)*16, ty*16..(ty+1)*16], and no two tiles share the same (tx, ty).

optimizations

Here are some additional optimizations that can be applied to make it fast for realtime use on CPU

fast approximate exp

the exp() function in the inner loop is by far the most expensive operation — it dominates everything else, if we don’t need 64-bit libm precision. the Schraudolph trick reinterprets a scaled float as an IEEE 754 bit pattern:

fn fast_exp(x: f32) -> f32 {
    let x = x.max(-87.0);
    let v = (12102203.0f32 * x + 1065353216.0) as i32;
    f32::from_bits(v as u32)
}

row-level early-out

in the tiled compositor, before entering the inner pixel loop for a row, we check whether the peak of the Gaussian along that row is below the visibility threshold. the Gaussian exponent along a row is a concave quadratic in dx:

the peak of this quadratic (at the vertex) is:

if even the peak is below ln(alpha_threshold / opacity), the entire row is invisible and we skip it:

let row_peak_cutoff = (alpha_threshold / opacity).ln();
let inv_a = 1.0 / a;

for py in y0..=y1 {
    let dy = py as f32 - sy;
    let row_base = dy_coeff * dy * dy;
    let row_slope = cross_coeff * dy;
    let row_peak = row_base + 0.5 * row_slope * row_slope * inv_a;
    if row_peak < row_peak_cutoff {
        continue;    // entire row is below threshold, skip it
    }
    // ... inner pixel loop ...
}

a subtlety: the peak is at the vertex of the quadratic, not at $dx = 0$. using row_base alone (which is the value at $dx = 0$) would miss peaks that sit off-center and could incorrectly skip visible rows for elongated, tilted Gaussians.

scratch buffer reuse

the radix sort needs an auxiliary buffer the same size as the input. allocating this every frame would be wasteful. instead we allocate it once and reuse it across frames via a ScratchBuffers struct:

pub struct ScratchBuffers {
    pub sort_aux: Vec<Projected>,
    pub tiles: TileBins,
}

the sort_aux vec grows to fit the first frame and never shrinks this enables us to do zero per-frame heap allocation once the system has warmed up.

The usual response I get when I bring this up with friends is that tools have always shaped the people using them, the printing press did it, radio did it, standardized spelling did it, twitter did it, so what is the big deal about this one. And I think the big deal is that the loop is closed now and the loop is short. Previous tools did not update, the printing press in 1500 was the same printing press in 1600, twitter is roughly the same twitter it was five years ago. But claude 4.6 is not claude 3.5 and it will not be claude 5 in six months, and every retraining cycle grounds on an updated human interaction, and the 2026 internet is written by humans who have spent the last two years being subtly finetuned by the 2024 version of the model, which was itself trained on humans that had been talking to the 2022 version. The model distribution and the human distribution are idk sort of converging, and maybe at one point the reference you would use to even measure the drift, stops existing because nobody is untouched anymore. I don’t know if this converges into something stable or diverges into something strange but either way the concept of “what unassisted human thinking sounds like” quietly diminishes, and I don’t think we are going to notice the day that happens

I heard the other day that people have stopped open sourcing work that they would have happily published a few years ago, specifically because they know it will end up inside a training corpus and the edge they got from writing it will evaporate. The funny second order thing here is that the models got good in the first place because people were generous in public, and the models being good is now the exact reason people are not being generous in public anymore. The commons is being fed into a thing that makes continuing to contribute to the commons feel like a bad trade, and I don’t see any version of that where the commons ends up richer

The other thought which I came across while writing this post is a quiet shift in what humans are actually for in this whole arrangement. I met a second year undergrad a couple weeks ago building a startup, smart kid, and he walked me through his workflow with real pride. He maintains this enormous knowledge base about his company and the agent acts as the CEO in his framing, and he is the hands, the agent drafts the email and he clicks send, the agent preps the talking points and he goes and says them to the investor, the agent reads the contract and tells him where to sign, and I could not figure out in the moment how to tell him that from where I was sitting he had just described himself as a peripheral, an io device with legs, a face and a bank account rented out to something else’s cognition. And the same shape is showing up in people who route feedback and delegation and even performance reviews through a model, so your manager is not writing your review and your response will also not be written by you, and the two models are going to have a little conversation about your career while the two humans pretend to have opinions in the meeting afterwards. The core limiting factor of most industries used to be intelligence, and the story of the last century is basically the story of that factor getting commoditized in stages, first physical labour, then access to knowledge, and now the parsing of that knowledge itself, which is the thing we used to call being smart. What you cannot hand off yet is the legal identity, the bank account, the face on the zoom call, the body that walks into the room, and so that is quietly what humans are being optimized into in this new stack. Not the thinker anymore, the thing that makes the agent’s output count as a real action in the real world, maybe humans will become just an extended version of tools to interact with the world

What if all this becomes the norm and we all shrug and get along with it. In an earlier post I wrote about asking claude to apply for my visa and I framed it as a fun anecdote because it genuinely was, but there is a darker version of that same story and it is the one where my juniors are firing up a claude and asking it to apply to 400 jobs a night while recruiters on the other side are firing up their own claude to screen the same 400 applications, and the only humans in the loop are the ones who wrote the prompts on either end. I already see it happening, my juniors are not landing interviews and I do not think they are less capable than I was at their age, the signal to noise ratio of the application pool has just collapsed because the noise is free to generate and the signal costs the same it always did. I do not want to live in a world where two models negotiate the hiring decision in the middle while the humans on both sides wait around to be told the outcome, and I am not sure if we can get a vote on this