Once we moved beyond individual cameras, we ran into a modelling problem. A camera by itself didn't tell us enough. We needed to know where it was, what role it served and what we expected it to observe.

We modelled this as Centre > Camera Position > Camera > Observation .

A centre represented the physical site and also defined the access boundary. A camera position represented the role a camera played, such as a pool deck or floor entrance. It captured details such as when to capture, how often to check and which part of the space it covered. The physical camera was then mapped to that position.

This model made observations reusable across centres. Common behaviour could be defined once, while camera-level overrides handled differences in angle, noise, or coverage. Rules were attached to observations rather than individual camera serial numbers, making onboarding easier to scale.

Enlarge diagram ↗

Manual CCTV monitoring and separate dashboards worked at smaller scale but became difficult to operate as the camera fleet grew. Creating an alert for every matching event also produced too much noise because repeated observations could represent the same underlying issue.

Video introduced another trade-off. Keeping continuous streams ready increased storage, transfer, and processing costs, while sending long recordings to an AI model increased latency without necessarily improving the result.

Ops Tower uses sampled frames for detection and retrieves full recordings only when an operator investigates an alert. Live video starts when a user opens a stream and stops when the last viewer leaves. This keeps media usage tied to operational need rather than to every camera running continuously.

Enlarge diagram ↗

Enlarge diagram ↗

Once detection was running, another problem showed up. Not every event needed someone to act on it.

An event was simply something a detector observed. A defect added a classification to that observation. An alert meant the issue had crossed the threshold for operator attention.

Enlarge diagram ↗

Kafka carries incoming events, while ClickHouse stores their history. Alert rules use that history to evaluate frequency, cooldown periods, quiet periods, and repeated occurrences without turning every observation into a separate alert.

Mute and suppression allow temporary or planned pauses without deleting the underlying event history.

The operator ultimately sees an alert with its state and context, rather than a raw stream of events.

An alert is useful only when it contains enough context to support action. Ops Tower links the centre, camera, timestamp, event details, snapshots, clips, live video, and nearby recordings to the alert.

Enlarge diagram ↗

Live video supports current observation, while playback supports investigation of what happened around the alert. Keeping both paths connected to the same camera and alert record prevents operators from reconstructing an incident across multiple tools.

AI could help interpret an observation, but the output could still be ambiguous. A model could also return an unexpected response or time out.

That meant the AI path needed some structure around it. The output was parsed into a predictable format before deterministic rules acted on it. When the model could not provide a reliable result, the system needed a fallback rather than treating the missing response as a decision.

Enlarge diagram ↗

The operator remained part of the process. An alert could be acknowledged, resolved or ignored. Feedback could also record whether the detection was correct.

This kept the model in the role it was good at: helping interpret the available evidence. The final action stayed with the operator.

Adding a camera touched more than the camera itself. The new device had to be mapped to the right position and observation. Its events had to reach the right rules. The media path had to be available, and its access scope had to be right.

The same applied to alerts. The relevant rules, notification subscriptions and monitoring had to be configured before the camera could become part of the operating system.

This made onboarding a platform concern. A camera could be physically installed in minutes, but Ops Tower needed the rest of its configuration to match.

That model helped avoid bespoke setups for every centre. Position defaults handled the common case. Overrides handled the exceptions. The result was a fleet that could share the same observation model without requiring every camera to be configured from scratch.

Ops Tower is now live in production across 10,000+ cameras and 1,000+ centres. Operators can work from a single alert flow, with the relevant centre, camera and evidence available when they need to acknowledge or resolve an issue.

The bigger change was in how the system could be extended. Camera positions and observations could be reused across centres instead of creating site-specific setups. Alerts could also be rolled up within the same model.

The result was a common operating layer for detection, investigation and response across the fleet.

At this scale, the observation contract became one of the most important parts of the system. A common vocabulary made observations and rules reusable across centres instead of tying every check to an individual camera.

The distinction between event, defect and alert became just as important. Keeping those jobs separate gave each part of the system a clearer boundary.

Real-time also had to hold across the whole path. Fast detection alone meant little if routing, evidence retrieval or notification introduced delays.

AI could help interpret evidence, but ambiguous results needed structured fallbacks and human review.

Ops Tower was also a cross-platform effort. The work brought together Nilesh Kumar Sinha, Ram Aggarwal, Utkarsh Purohit, Vedant Mishra, Vidyakant Dubey and Pooja Srivatsav from the Data Platform team, alongside the Ops team. Different parts of the system had different constraints, but they had to come together as one operational path.

If we built Ops Tower again, we would instrument production latency and cost from the beginning. Those measurements are much harder to add once the system is already running.