This year’s event was promoted and run by CCTV In Focus SA – CIF SA – who became our official agent in March.
Rotatest Compliant Services (PTY) Ltd, trading as CCTV In Focus SA, has certainly had a great start to its first year.
Employing the audit and testing method of the Secured by Design licensed UK company CCTV In Focus, CIF SA secured an audit contract on behalf of SAPS (the South African Police Service) at a national key point. As a result, a complete system re-design was required, including an increase in cameras and improvements in system extent to cover the blind spots.
CIF SA worked together with the appointed project installer, and the complete system has now been installed, tested and commissioned to the Rotakin standard ensuring that this key point now produces images that are ‘fit for purpose’.
A full audit has also been carried out on behalf of Public Works (the South African Home Office) at another national key point resulting in the improvement of overall image quality produced by the system. The client was delighted with the results and, as a consequence, the company has developed a good working relationship with SAPS under the team leadership of Lt-Col H J van Heerden of the Standards and Specifications Security Advisory Service at Protection & Security Services (who also helped in putting together the guest list for our second annual CCTV ‘shoot-out’ which took place at Monaghan Farm near Lanseria).
2011: an extremely busy year
As stated, 2011 has been productive for both CCTV In Focus UK and our South African agent. At the UK head office, we’ve been busy testing cameras on the London 2012 site as well as bringing Travelodge Hotels on board as a new client.
Importantly, we’ve further developed the Rotakin test target such that it now includes a colour band, number and text testing as well as a circular resolution chart which has been used to test depth of field.
The redesigned target has been registered as Digikin (and this is now a recognised trademark).
The Digikin package being marketed in South Africa includes the Rotakin/Digikin and HOSDB face charts, and has been recognised as the testing kit to be used by SAPS (the first organisation to purchase units and undertake training in South Africa conducted by CIF SA).
This package has now been sold to many installers and suppliers and is proving to be an effective tool in the correct establishment of CCTV systems.
CIF SA is currently in discussions with companies including ABSA, Liberty Life and ESCOM on how to improve the quality of CCTV images from their systems.
Testing conditions for the ‘shoot-out’
Anyway, this year’s CCTV ‘shoot-out’ (as it’s known in South Africa) conducted in the week beginning 25 July, which is winter time on the other side of the world, remember!
The chosen site – the aforementioned Monaghan Farm – is about 15 miles south of Johannesburg and consists of open farmland. The testing site location is in a shallow valley where, during July time, the temperature ranges from -4 to +20 degrees (in other words a big problem for your body let alone CCTV cameras!).
A strict camera test procedure was defined a few weeks in advance of the day and agreed, with the four companies attending the event (namely Axis Communications, Norbain SA, Sentronics SD and Elvey/Pentagon).
All were taking part for the second year in succession. Last year saw the introduction of Rotakin to the country, this year it was ‘down to business’ with Digikin.
All four companies selected a wide range of cameras in order to test their real capabilities.
Axis Communications wanted to test its P1343-E, P3367-VE, P1346-E, Q6035-E, P1347-E, M114-E, Q1602-E, Q1921-E, P3344-VE and Q1755-E models.
Norbain SA was looking to test the Samsung .50 A1 low light camera, the Sony .50 CCD with Exwave HAD technology analogue camera, the Hikvision 480 TV lines .25 Sony CCD (3-88 mm) model, the Sony V-Series HD (720p) network camera and the Sony V-Series Full HD (1,080P) network model.
Also on test were the Pentagon-Bosch NBN-498, GEN5HD, NBN-921, GEN4 300 Series 36x, NBN-832, EX30NG IP and Junior HD cameras.
Last, but by no means least, also on trial were the Sentronics-Pelco EE20 Series vandal-proof models, IP-8151, IP-8161, the IP-8362, the Pelco Spectra IV IP dome (x35) and the Pelco IXE20 Series network cameras.
Tight timescales for the camera providers
The camera providers had 24 hours to set and adjust the cameras, all of which were installed at a height of 1.8 metres. Recorders were set at 2 CIF @ 6 frames per second (fps) with live images viewed at 25 fps. All suppliers had to provide the same size 22″ monitors for live viewing.
Each camera was set with a 2.8-8 mm lens for close distance testing and a 5-50 mm lens for long distance testing.
The set-up started early on the Monday morning, with cable runs and camera set-up being completed by late evening (apart from one supplier who had opted to run Cat 6E, and spent a long time working out why there was no video signal at the business end) As it turned out, the cables had become ‘pinched’ and twisted in the stand, so a few hours were spent re-running Cat 5E cable as a replacement.
The following morning – 26 July – everyone was ready to start testing, the target’s having been placed at a distance of 8 metres from the cameras. The testing was conducted as follows:
Some very interesting results were observed. The test site is east-west facing, so early morning testing has the sun coming up behind the camera and reflecting off the target.
The next test was conducted mid-afternoon, the test target being set at a distance of 30 metres from the camera. The following testing was conducted:
Each supplier was assigned an evaluator from our team. Using the target the live images were then scored for correct screen height, resolution, quality of text and numbers, detail of colour and depth of field.
We also used HOSDB face charts. Again, this was scored as suggested by HOSDB (or CAST as it is now known). We also conducted face chart testing (50% screen height). The scoring on this was conducted so that if the row could be identified, this would score as a pass. The last part of every test was conducted with targets rotating.
All the above testing was also carried out during the hours of darkness. This being South Africa in winter and away from any other light sources the average lux levels over the two nights of testing was found to be 0.03 lux.
In addition, testing was conducted using infrared lighting and white LED lighting.
The camera iris: responding to light changes
Part of the night testing included the evaluation of how the iris of the camera responded to changes in light. This was conducted using a high-powered flashlight pointed directly at the cameras. In this test the standard analogue cameras performed better than the megapixel and HD cameras.
Also new this year was the introduction of a megapixel/HD test chart purchased from VIDI Labs in Australia. Not sticking to the test process suggested by them, we placed the chart on the target at a distance of 8 metres and obtained 100% screen height, sticking to the screen height of the target as per PAL (this was to try and make it equal for all cameras).
We could have reduced the height to match the recommended figures as supplied by HOSDB Operational Requirements Manual 5/2009 for the 720p and 1,080p cameras, but after general discussions the 100% figure was agreed upon.
This test chart is normally used in lab conditions so it becomes very interesting to see the results under normal day light conditions and at a distance.
To make the event even more interesting, during the course of the day test targets were placed around the area in full camouflage in the undergrowth and left rotating. The suppliers were tasked with constructing search patterns with the PTZ cameras with the aim of finding the targets.
Many hours of entertainment were had by all, but in the end the targets were found, the furthest distance being just under one kilometre away.
Everything was OK during the day, but night-time realised a different scenario: not one camera found a target. This is where we started to look at the thermal imaging cameras. No problems there.
To add to the excitement, SAPS deployed four of its people to the undergrowth and constructed a commando-style raid on the site. The thermal imaging cameras were outstanding, even picking up the local wildlife grazing as the subjects neared the site. The ‘commandos’ duly released smoke grenades: no problem for the thermal cameras, but the conventional cameras had severe problems.
Reviewing and evaluating recorded footage
Thursday and Friday of the test week were open days for invited guests to attend. This gave the suppliers the opportunity to demonstrate their equipment under test conditions and talk to key clients.
The guest list read like a ‘Who’s Who?’ of the security industry in South Africa and Government departments, including representatives from SAPS, ACSA, Transnet, UNISA, ESCOM, the Reserve Bank, the Government Print Works, SADPMR, the NPA, Xtravision, SAPS Hawks, Petro SA, NECSA, Bombela and SABS.
The general conclusion from all was that it was an extremely worthwhile event to attend.
Next came the hard work of reviewing and evaluating the recorded footage, snapshots and motion footage that was downloaded by all suppliers.
In total, 40 gigabytes of footage was retained. As always there were problems with different players and formats, but these issues were overcome and the work began.
The results are now in and have been fed back to the suppliers. They’re not for general release, but will help the manufacturers involved assess the weak points within their systems.
Basic conclusions to be drawn
Our conclusions from the results are fairly straightforward. Megapixel and HD cameras are great if the subject is not moving. Be cautious when using high megapixel cameras on a moving subject as the subject can start to look like a banana (check out the image at the foot of this article).
Motion blur is more exaggerated during low light and night-time conditions. Beware of infrared and white light being too powerful: this just tends to white out the subject so you just see lots of ‘ghosts’ on the screen.
One last word of warning: never eat a BBQ cooked in the dark!
According to the majority of people on site, both sellers and buyers, the event proved a success, demonstrating what can and cannot be done with different surveillance cameras.
Norbain SA said such events provide an opportunity to test cameras in an environment to which distribution companies and manufacturers do not always have access. “However, these were the same tests and environment as last year [the first time this event was held in South Africa] and although a great idea, the next session should be shorter as providing all the necessary resources for a whole week is a problem.”
Norbain SA also suggested future tests ought to follow a specific approach, such as testing in a retail environment, including the options of high and low-end cameras. “This will prove of interest to a specific market segment.”
Norbain obtained very relevant results about how the cameras it sells perform in extreme conditions, such as various lighting conditions, bundled with the results from the Rotakin tests.
“In our opinion, Rotakin demonstrates the difference between installing a camera and configuring a camera.” Simply installing a camera focused on a specific area without verifying what the requirement is from the client often leads to trouble as the camera may not be able to deal with real issues such as verification, identification and constant movement, etc.
“Rotakin is worthwhile from a distribution and manufacturing point of view, and more important for installer training purposes. Its relevance to end users is questionable.”
Rotakin target: adoption as standard
Elvey, the company that recently acquired Pentagon, stated that the event was “very professional and worthwhile” and that it “showed a different level of competency from the companies that play in that market space. The Rotakin target is great and, hopefully, is adopted as the standard so that all future tests are conducted from the same platform, thus levelling the playing field.”
Jason McGregor was on site on behalf of Axis Communications. He found it more focused and organised than before.
“We definitely feel the event is a great opportunity for vendors to field test the equipment in challenging environments and determine their real life limits,” said McGregor. “A huge Thank You must go to Ernst and his team from CCTV in Focus.”
Concerning the results, McGregor said: “We found that, in general, our cameras showed outstanding performance in all forms of testing with depth of field, resolution and colour reproduction being the main strong points. The Rotakin standard definitely has a place in our market and, if employed correctly, the benefit will be realised by manufacturers, resellers and end users alike.”
He added: “The Rotakin not only tests the camera but incorporates operator, camera and system testing, in turn ensuring a comprehensive audit on all faculties involved.”
Of interest this year, McGregor commented that SD card (on board storage) support is being included by more manufacturers as well as the NVR vendors, who are developing their software to index and incorporate SD cards as part of the recording database (and thus alleviating strain on servers and the networks). A direct result is less traffic on the network and lower requirements on server hardware.
More than a CCTV camera testing tool
Gemma Forsyth was there to represent Sentronics. “The event provided us with an opportunity to showcase some of our latest camera technology and verify the capabilities of these devices,” explained Forsyth.
“Cameras perform differently under certain test criteria, and it is a good way to position our products, not only from a performance perspective but relative to pricing as well.”
Forsyth stated: “The time and effort was definitely well spent and I’m sure we will attend again next year, by which time new products will have been released and we will have the opportunity to test and showcase these new technologies.”
In terms of this year’s results, Forsyth explained: “We saw some very favourable results from our cameras and were pleased with the performances, especially in the low light tests. The Rotakin testing system provides more than a camera testing tool as any full system weaknesses can be exposed and corrected. There’s definitely a requirement for this kind of testing.”
As a supplier, Forsyth’s organisation is always faced with system integrators claiming equipment faults or non-performance when in fact it’s usually just a question of the equipment being installed in the incorrect manner.
“With the Rotakin system, we can ensure that our systems are not to blame and that the equipment we sell is fit for purpose,” urged Forsyth. “These tests also provide the client with piece of mind.”
Forsyth also said that Sentronics has seen a growing interest in intelligent video and open platform solutions of late.
“The market seems to be looking very hard at the benefits of analytics coupled with the quality of HD video quality and edge recording.”
Planning underway for Year 3
So a successful event. Planning is already underway for next year’s tests. We’ve had more suppliers asking to take part, and the plan is to try and gain an empty building and construct internal and external testing set-ups for real world testing.
CCTV In Focus SA has already carried the testing of CCTV equipment one stage further by constructing a test lab at its premises, testing over a distance of 10 metres and with variable lighting down to 0.1 lux.
Indeed, the team has been very busy testing various pieces of kit. There has also been a requirement from SAPS for testing of kit to meet minimum specifications as part of tender processes.
We strongly believe that our colleagues in South Africa are in a great position to help avoid the pitfalls that we made (and are still making) here in the UK. Time and time again we visit sites in the UK that have been passed for the British and European Standards but have missed the basics of an operational requirement: no camera has a defined purpose and the quality of recorded images has been far from ideal.
The systems have been passed ‘fit for purpose’ on the quality of the live images not the recorded.
When will end users wake up to the fact that it’s the recorded image that’s the key part of any system? After all, you cannot replay a live image in court.
Andy Hays is the operations director at CCTV In Focus UK
