Shooting & editing HDR via Avid using CLog gamma

We've got BVE2016 coming up and one of the things Root6 will be showing is an HDR workflow via Media Composer using Canon monitors.

HDR is still a bit of a crap-shoot as far as standardisation is concerned with the BBC/NHK system, Dolby Vision, Sony's SLog3 and Canon's camera-native CLog. The principle of using an alternate gamma so that you concentrate the bit-depth where you want the extra range is well established;

The hope is that all of these manufacturers will coalesce around ST.2084 which (amongst other things) defines how you handle the specular highlights; those very bright parts of the picture which give a real addition to the look of the pictures. These are typically defined to be >500 Cd/m² which is MUCH brighter than broadcast white! The idea is that the last bit of dynamic range (10th bit - all values above 512) represent the highlights and everything up to 50% is akin to the usual video dynamic range. You calibrate the monitor such that 50% is set at 100Cd/m² and just hope that the colourimetry of the highlights tracks RGB-wise!

So - Root6's own Dave Skeggs and I set off around Soho and London Bridge to capture some night time and daytime footage. We were using a Canon C300 mk.2 set to UHD (3840 x 2160) at 25P (no interlaced fields at UHD and no high framerates at that resolution unfortunately). We set the colour space to an optimistic Rec.2020 and gamma to Clog. In that mode the camera shoots 410mBit/s XAVC codec MXF files.
We've been using Media Composer v 8.5 on an HP Z840 workstation & the new Avid/BlackMagic DNXio video hardware; we had to update the firmware to get it to generate quad-link SDi. Although HDMI works it is nobbled down to eight-bit and so would not be suitable for this test. I would put a link to the video but none of the video sharing sites support HDR and neither does the screen of your tablet/laptop/TV! I took all the monitor photos with my Fuji bridge-camera in a very bright office; you'll have to take my word for it!

Notice the headlights of the taxi - you can see details inside the light!

exactly the same frame; notice the dark details in the trees against the night-sky.

 Of course on Media Composer's GUI display you get the CLog gamma rendered as if it was Rec.709 and so it looks very washed out and lacking in detail

You can have Avid flatten the gamma of source clips so that it looks OK on the GUI - that doesn't affect sequences that the clip mas been used in.

 

Quite a large range of alternate gamma and colour spaces

 It shows up in the bin-view which is useful

 

So now clicking the source window and setting the monitor to regular HD gamma (BT. 1886 fact fans) shows you what the same material shot on a "regular" camera would look like; very little detail in the blacks and none in the whites.

 

 Root6's own DOP; Dave "is that in focus?" Skeggs

I'd forgotten how limited a normal video-camera's dynamic range was. The Canon monitors top out the specular highlights at 400Cd/m² which is somewhat less than a Sony BVM-X300 (1,000 Cd/m²!) but for €10k less than the Sony (and losing only a stop-and-a-half of specular highlights) the Canon 30" UHD/4k IPS panel represents superb value. I was a bit disappointed that the camera tops out at 29.97P at >2k resolution so I couldn't see how nice fluid video motion looked at high res; everything has a jerky film-look to it.
Steve Shaw at Light Illusion has a very good article exploring some of the fundamentals of HDR.

Reliable Avid ethernet traffic over old fibres

The first Avid Unity fibre-channel SAN I installed was in 1999 and fibre was the standard for high-speed shared storage for editing for more than a decade. However; since the introduction of MPEG4-based editing codecs which allowed Avid to offer high quality DNxHD we've been all about ethernet-attached shared storage; in Avid's case the ISIS storage products.

We have had a lots of customers who went to the expense of running OM3 (50 micron multi-mode fibre) cable only a few years ago and are now cheesed-off that they need to flood their facility with cat6 or 7 because the rough-old cat5e they have doesn't work reliably for gigabit. So; the obvious choice is to use that fibre which had bags of bandwidth for ethernet but until very recently that was not an approved configuration by Avid. Recently they have tested Allied Telesis media converters and given a cautious thumbs-up. BUT, they are very expensive (a few hundred quid per workstation, so back to running new cable), but I had a quick chat with the guys at Comtec about the house-brand they supply which uses the same chipset.

Neal Kemsley kindly ran Avid's PathDiag tool before and after and these little cheapies seem entirely transparent. Here's what he fed back to me;

"Windows ISIS client attached directly to a Dell N3048 switch running Pathdiag in an “Unlimited” Writes then Read PathDiag test cycle targeting an ISIS Workspace (the term used by Avid for their storage volume – the same as Unity). Unlimited means that the test attempts to saturate the channel between the ISIS client and is a good indicator of the upper limits of the potential connection between the client and the storage. In this case it is a 1 Gbit copper connection to the switch, and an optical 10 Gbit connection between the switch and three ISIS 5500 storage chassis. As you can see we are getting a result north of 100 MBs per second – yes that is MegaBytes – not bad for a 1 Gbit path! (Not too shabby as they say in Boston!)"

"This one shows the same test parameters but with the client attached to the switch using the media converter pair in circuit. Note that the test results are really similar perhaps with some minor variation in the upper reaches of the test during the write cycle but nothing serious. The overall speeds achieved during the tests are essentially the same as a direct connection and there are no errors displayed.
Note that the graph pattern drawn by the test is relatively clean showing very little spiking or variation and clean transitions between the Write tests and the Read test cycle. Note also that no errors are seen in the error count on the right side."

"We would want to run the test for several hours to draw conclusions on this but these results are very promising. We would probably want to also change the Transfer Size parameter of the test up and down to emulate different editor timeline characteristics. Smaller values are used to emulate working with heavily compressed material, the current setting being used to emulate working with DNxHD material, and larger values can be selected to emulate working with uncompressed HD and UHD material."

"To emulate working with several streams of data in a timeline this shows four independent PathDiag test sessions running simultaneously with the Media Converters in circuit. In this case rather than working with unlimited tests, I set the Transfer Rate parameter to 25 MB/sec and allowed the tests to cycle for 30 minutes. Notice for results graphed in these tests the individual tests are interacting somewhat – see that the top value levels are becoming choppy and castelated somewhat as each test competes for throughput. Since the tests in aggregate are pushing the maximum limit of the channel (if all four tests happen to be writing or reading simultaneously, the overall write or read bandwidth should around 100 MB/sec) this interaction is quite normal and will get worse if similar test were being run on further clients in the ISIS environment as each client competes for access to the storage."

Why do manufactuers over-specify power requirements for broadcast equipment?

It's actually a rhetorical question and I'm glad they do. Most of the time I have to tell a customers' electrician and air-con contractor how much power (and hence how much heat) the machine room will be pulling/genarating. Most customers refuse to believe that 99.9% of the electrical power entering a server room/TV MCR leaves it as heat! Just think about it; a 1v video signal leaving the room and terminating into 75 ohms represents a tiny amount of energy. Everything winds up as heat and so I've got to the point where I tell the electrician how many amps we'll need and the aircon guy how many BTUs of heat he'll have to move. By turning them into different units the customer stops complaining!

Anyway - why are the numbers always so different? I've been installing Avid shared storage chassis since 1999 when Unity v 1.2 was considered clever - 500 Gigs across three arrays and usable by around ten edit rooms. Fast forward to 2014 and the ISIS range are what you'll buy from Avid and the new ISIS 2500 near-line storage is just the thing for cheaper, non-edit storage.

This is the rear of this monster - two supplies with 20A C19 inlet connectors and you can see from the clamp-meter that the thing is pulling 1.3 - de-powering one of the supplies shows the current draw by the single supply rise to 2.6A (so they are properly balanced). Re-powering the thing shows that the total draw across both PSUs rises to 3.3A for around thirty seconds but settles to the total 2.6A once everything is up and running. 

So, P=IV and (not forgetting the inductive load which has a power-factor of 0.8) means we are seeing a bit less than a kW max. However - on the Avid website;

 

Avid, KVM systems and the cry of "...not fit for purpose"!

There are several things that happen when you plug a monitor into a graphics card. I'm assuming DVI (this is the 21st century!) and all other displays standards; HDMI, DisplayPort and Thunderbolt all follow a similar principle.

1. Pin 16 on the DVI connector on the graphics card is refereed to as the "hot plug detect" pin and is held logic-high (at +5v) through a very high value resistor. When you attach a monitor the pin is momentarily taken low to alert the graphics card to the fact that a monitor has been connected. 
2. The graphics card generates an interrupt on the PCI-e bus
3. Windows sees this and uses it to generate an EDID exchange request
4. The monitor responds with it's EDID profile
5. If the EDID profile is the same as last time the graphics card driver does nothing; it's the same monitor after all
6. If the EDID is different the driver re-sets the system resolution - this is particularly important if it's either a lower resolution or frame rate than it was running at before; if it didn't do this you'd get black screens. Nobody wants that.

Now then, Avid Media Composer does something different! It listens for the interrupt and when it sees it it halts operation and displays an error message saying it needs to re-start. Even if all that has happened is that your monitor cable has fallen out of the back of the computer and you've reconnected it - go figure.

The upshot of this is that when you're using a KVM routing solution of any kind and you assign a new pair of monitors to the Avid it insists on re-starting. I've been aware of this for a while and I let people know about it when demo'ing or presenting at trade shows; but it's just the way of things. There is nothing you can do whilst Avid does the wrong thing. Amulet - my favorite KVM-over-IP system does exactly the right thing; it holds off asserting pin-16 (and triggering the chain of events above) only when absolutely necessary. A media operator can be switching around four Media Composers and each workstation is unaware that the operator is being promiscuous. Amulet only asserts pin-16 when there really is no other option; when a new desk-end "zero client" takes control of a machine it hasn't yet seen. The only place I've seen this to be a problem so far is when an editor starts a layback to videotape, presses disconnect on his desk-end zero-client and calls his operator saying "...I've started the layback, I'm off home; can you watch it through to the end?". Then, the operator tries to acquire that Avid and by necessity the Amulet has to alert the computer to a new pair of monitors and Media Composer halts (ruining the layback).

This was the issue at ITV Salford and I home-brewed a little gadget to stop the Avid being able to tell when new monitors where attached; I essentially neutered it's ability to detect pin-16. 

So, I think Amulet does exactly the right thing, if it didn't you'd loose the ability for workstations to detect what monitors were attached and pretty soon you'd have rooms where you had black screens; nobody wants that. The bogeymen here are;

• Avid - why on earth doesn't it do what Windows does?
• Editors who don't watch their own laybacks!

We are now in an argument with a customer who I explained this all to when I was demo'ing Amulet, but we didn't win the SI quote, but we still supplied the KVM. The SI who is installing is shouting blue-murder about "...not fit for purpose" and we're having to brew up 150 adaptors to keep everyone sweet. My feeling is this will cause more trouble further down the line just for the ability to not close an Avid project for those occasions when you need to hand a machine off to someone else. What kind of workflow needs that?!

Fixed the DVI / pin-16 hotplug dilemma

As with a lot of mod'ing or (dare I say it!) bodging solutions you need to find a nice connector or pre-made cable to base your fix on. If you look back at the problem we've been having with Media Composers switched across different Amulet heads then you'll recall it wasn't an EDID issue, rather one of Windows detecting a monitor change; Amulet does the right thing, it's Avid that's the problem.

The fix is easy; you need to tie pin-16 (hot plug detect) to Vcc (+5v on pin 14) via a 1K resistor;

The best mod'able pre-made cable that is suitable is one of these from Lindy.  Now I just need to knock up a dozen for the customer!