Redundancy

The main PC dies at 20:41. Nobody in the stand notices.

Every club asks this in the first call, and most software answers it with the word "redundancy" and no detail. Here is exactly what happens, in the order it happens, and what it needs from your rack.

The shape of it

Four numbers, and the gap between two of them.

1 s
heartbeat between the two machines
~5 s
of silence before the backup takes the wall
1 frame
the transition itself, with no black
2 machines
one project, pushed one way only

What it actually is

Not a spare machine in a cupboard.

A spare PC with the software installed is not redundancy. Somebody has to notice the wall has gone, find the machine, open the project, work out where the rotation had got to and start it, and by then the half is over and the sponsor has lost its seconds.

A USM backup PC runs the whole match alongside the main one. It holds the same project, it follows the same cues, and it knows which slot is on the wall right now, because it has been following all along rather than waiting to be woken. What it does not do is drive the wall, until the moment it has to.

The two machines talk on the venue network once a second. That is the whole failure detector: not a cloud service, not a person, not a monitoring dashboard that emails somebody. If the main stops answering, the backup already has everything it needs to take over, and it does.

The switch, second by second

What a dead PC looks like from the stand.

An illustration, not a recording of one night: the times are the intervals the software actually works to, laid on a clock so the gap is visible.

  1. 20:41:02

    The main PC stops

    Power, a kernel fault, a pulled cable. From the wall’s point of view the picture is still there: the LED processor is holding the last frame it was sent.

  2. 20:41:03

    The first heartbeat is missed

    The backup was expecting a beat and did not get one. One missed beat is not a failure; a machine can be busy for a second.

  3. 20:41:07

    The backup takes the wall

    Five seconds of silence is the threshold in the code. The backup promotes itself and drives the wall. It has been mirroring the master’s cues all match, so it knows the playlist and where the match had got to rather than starting cold.

  4. Half-time

    Somebody deals with the main PC

    Handing the wall back is a decision somebody makes, never automatic. Two machines both believing they are in charge is the one failure worse than the first, so USM makes an operator say which is which, and warns loudly on both screens if it ever sees two.

The parts that catch people out

What we would ask you about your rack.

  • Both machines need to see the processors. The backup drives the same LED processors as the main one. If only one PC is cabled to the wall, it is a spare, not a backup.
  • Several screens stay together. A ground with a perimeter run and a fascia does not fail over one screen at a time. The mirroring is by position, so screen two on the backup is screen two on the main.
  • The project is the same project. Both machines take the same project from the cloud, so a change made in the office an hour before kick-off is on both. A backup running last week’s sponsors is worse than no backup.
  • Put them on different power. Two PCs on the same extension lead fail together. UEFA asks for dual signal servers each on a UPS for its own competitions, and that is good advice for a league fixture too.
  • The licence covers it. A backup PC is an add-on to the same licence rather than a second full one. It is priced on the pricing page with everything else.
  • It does not make the wall immortal. This protects against the machine that plays the wall. A dead cabinet, a cut cable or a processor that lost its input is on the other side of the DVI socket, which is what LED monitoring is for.

What it takes

The setup, in one table.

Machines
Two Windows PCs, both cabled to the LED processors
Network
Both on the venue LAN, no internet needed for the failover itself
Heartbeat
One second, machine to machine
Failover
About five seconds of silence, then the backup is on the wall
Switch back
Manual, between halves or after the match, never automatically mid-fixture
Screens
Mirrored by position, so multi-screen grounds stay in step
Licence
A backup PC add-on on the same licence
Power
Separate circuits, and a UPS each if the fixture is a European one

Being straight about it

What we have not solved.

There is a gap, and it is about five seconds long. If a sponsor rotation was due to change inside it, it changed late. We would rather write that down than describe a switch as instant.

We could make the gap shorter by treating a single missed heartbeat as a failure, and we do not, because a machine that is briefly busy would then hand the wall away in the middle of a match for no reason. Five seconds of a held frame is the cheaper mistake.

And the switch back is deliberately yours. Software that fails back on its own would produce a second transition at a moment nobody chose, usually while somebody is still working out what happened to the first machine.

Ask us what happens in your ground.

Tell us how your two machines would be cabled and powered, and we will tell you honestly where the weak point is, including when it is not the software.

Talk to us