The “cheap” Test Pattern Generators thread
Posted: 20 Aug 2019 22:54
Finally I could satisfy my (and I hope your) curiosity to know which test pattern generator (or TPG) in the low price range is really accurate. To do this I bought a medium-high end TPG: DVDO AvLab TPG, which has inside a signal analyzer whose accuracy has been verified and confirmed with an Accupel DGA-6000 by the legendary Maciej Koper.
Let's start! Here are the contenders:
Apple TV 3rd Gen (Model A1427)
Apple Media Center of two generations ago. Can be used with MobileForge (CalMAN) and LightSpace Connect (LightSpace). The limit of this object is the adjustable refresh rate, which is limited to 50 / 60hz, no 24hz.
Apple TV 4K
The latest Apple release. Can also be used with MobileForge (CalMAN) and LightSpace Connect (LightSpace).
On the spec sheet a bomb...
Amazon FireTV Stick
Very economic and portable object, with a more than decent video quality. Can also be used with MobileForge (CalMAN) and LightSpace Connect (LightSpace). Here the limit is that the output is only one: RGB limited (16-235), no YCrCb
Raspberry Pi 3 B + with PGenerator by Riccardo Biasiotto
A tool that can really do everything! The only limit is the imagination!
Is my laptop bit accurate? Dell Inspiron 17-7779
...and my Blu-ray disc player?!? Panasonic DMP-BDT260
Before moving on to the test results and their commentary, I would like to define what the reference TPG should be... easy, it has a name: Accupel DGA-6000! It is perfectly accurate in every color space (supports up to Rec2020), with every color depth (8/10/12 bit), at every resolution (up to 4K) and frequency (24/25/30/50/60), both in SDR and HDR. It has one single flaw: it lacks of Dolby Vision support, but we know whose fault it is!
Limiting the test to SDR, I can state that a reference TPG must be bit accurate in YCbCr at 1080p resolution with all refresh rates, in this case the reference color space is Rec709. It is therefore in this "enclosure" that I have carried out my tests. Where a device resulted not accurate in YCbCr (or the YCbCr output setting is not present) I have tested its accuracy in RGB Full or Limited but with 16-235 videoscale. The display used is a TV Monitor Samsung SyncMaster P2370HD which, despite obsolescence, it has played its irrelevant task very well.
How I did the tests: DVDO AVLab TPG powered via USB from the laptop, TPG under analysis connected (one at a time) to the HDMI input of the DVDO with the HDMI output connected to the display, CalMAN 2018 Ethusiast R3 Beta Build 27, LightSpace CMS HTP 9.0.0.2448.
TPG for CalMAN:
Ted's Disc Media Files with third generation Apple TV
YCbCr 444 8 bit
MobileForge with Apple TV third generation
YCbCr 444 8 bit
RGB High
...and RGB Low
MobileForge with Apple TV 4K
YCbCr 444 8 bit
MobileForge with Amazon FireTV Stick
RGB Limited 8 bit
RPi PGENERATOR by Riccardo Biasiotto
RGB Full 8 bit
11.21.2018 UPDATE: As requested by [MENTION=8489745]ebr9999[/MENTION] I tested RGB limited output of RPi PGenerator. As previously stated, the not so bad results still prove that RPi is bit accurate only with RGB Full output with LightSpace or CalMAN videoscale set to 16-235.
Here it is:
END OF 11.21.2018 UPDATE
06.24.2019 UPDATE: As requested by [MENTION=8489745]ebr9999[/MENTION] I tested RPi PGenerator YCbCr limited output, videoscale set to 0-255. Again, as stated above, the results still prove that RPi is completely bit accurate only with RGB Full output with LightSpace or CalMAN videoscale set to 16-235. But, what is better for you, it depends on your TV RGB ---> YCbCr rounding errors.
Here it is:
END OF 06.24.2019 UPDATE
Dell Inspiron 17-7779 with CalMAN built-in TPG
RGB 8 bit
Ted's bluray disc with Panasonic DMP-BDT260
YcbCr 444 8 bit
Conclusions: As shown by the graphs, at the moment, MobileForge is not bit accurate with every device even if the output of the latter is (as in the case of Apple TV third generation, as proven by the test with the media files of Ted's disk). MAYBE wrong triplets are sent. With Apple TV 4K the situation seems to improve a bit but it's just an illusion. The Apple TV 4K, in fact, with output YCbCr and RGB seems to process the image continuously returning fluctuating values with digital errors ranging from a theoretical minimum of 1 to a theoretical maximum of 6 per triplet (as shown in the chart in which Ted patterns are used). For this reason, even if I reported the minimum values (by looking for them pixel by pixel) I did not calculate the relative dE for a simple reason: Apple TV 4K is not effectively usable as a TPG. That's why I will not proceed to test it further.
11.13.2018 UPDATE: Obviously, if ATV 4K is the only TPGable device you own you have no choice than use it. My only recommendation is to NOT measure the patches produced by it in contact mode, the more patch area you measure the more the digital errors will be reduced... forewarned is forearmed.
Since there are still some doubts about ATV 4K behavior with YCbCr/RGB high output, I filmed two videos, both displaying Ted's Color Checker Classic Media file via Infuse 5 Pro App with:
1) ATV 4K at 1080p 24 Hz YCbCr output
2) ATV 4K at 1080p 24 Hz RGB high output
NOTE: Except Rpi, I tested all devices with Ted's Color Checker Classic media files (or BD for my BD player) and all of them showed solid triplets except ATV 4K.
A very nice request to analyze CalMAN built-in with my laptop HDMI set to YCbCr has been made by baller02, I didn't think about it when I ran all the tests so I've analyzed Calman built-in TPG only with my laptop HDMI set to RGB and it turned out to be bit accurate. So prior to analyze CalMAN internal TPG in YCBCr, I have to do it with Ted's patterns and... Read this post for more.
END OF 11.13.2018 UPDATE
The Raspberry Pi (RPi) PGENERATOR by Riccardo Biasiotto is bit accurate and very stable (in RGB Full, not in YCbCr). We're talking about a DIY solution made by a NON PROFESSIONAL. Really outstanding, especially because it is 360° configurable, it connects via Ethernet, Wi-Fi, Bluetooth and it works with every calibration software I know!
Surprisingly, the HDMI output of my laptop resulted bit accurate!!!
TPG for LightSpace
LightSpace Connect with Apple TV third generation
YCbCr 444 8 bit
LightSpace Connect with Amazon FireTV Stick
RGB Limited 8 bit
RPi PGENERATOR by Riccardo Biasiotto
RGB Full 8 bit
Conclusions: Apple TV 4K has been excluded for the reasons mentioned above. All TPG solutions for LightSpace are bit accurate. The choice here could fall on the third generation Apple TV which is also perfect in YCbCr. On the same level, the RPi PGENERATOR by Riccardo Biasiotto.
Final Conclusions and tips: For those who only use CalMAN or even LightSpace, the only affordable and accurate bit solution is the RPi PGENERATOR by Riccardo Biasiotto.
For those who only use LightSpace and own a third-generation Apple TV, they can download and purchase LightSpace Connect but it is strongly suggested to use it connected via an Ethernet cable. Wi-Fi, during the generation of a color cube, could disconnect making you lose happy hours of real life. The RPi is also highly recommended with a super stable wireless Bluetooth connection.
Please, let me know your opinions, suggestions and (why not?!) your own tests!
Let's start! Here are the contenders:
Apple TV 3rd Gen (Model A1427)
Apple Media Center of two generations ago. Can be used with MobileForge (CalMAN) and LightSpace Connect (LightSpace). The limit of this object is the adjustable refresh rate, which is limited to 50 / 60hz, no 24hz.
Apple TV 4K
The latest Apple release. Can also be used with MobileForge (CalMAN) and LightSpace Connect (LightSpace).
On the spec sheet a bomb...
Amazon FireTV Stick
Very economic and portable object, with a more than decent video quality. Can also be used with MobileForge (CalMAN) and LightSpace Connect (LightSpace). Here the limit is that the output is only one: RGB limited (16-235), no YCrCb
Raspberry Pi 3 B + with PGenerator by Riccardo Biasiotto
A tool that can really do everything! The only limit is the imagination!
Is my laptop bit accurate? Dell Inspiron 17-7779
...and my Blu-ray disc player?!? Panasonic DMP-BDT260
Before moving on to the test results and their commentary, I would like to define what the reference TPG should be... easy, it has a name: Accupel DGA-6000! It is perfectly accurate in every color space (supports up to Rec2020), with every color depth (8/10/12 bit), at every resolution (up to 4K) and frequency (24/25/30/50/60), both in SDR and HDR. It has one single flaw: it lacks of Dolby Vision support, but we know whose fault it is!
Limiting the test to SDR, I can state that a reference TPG must be bit accurate in YCbCr at 1080p resolution with all refresh rates, in this case the reference color space is Rec709. It is therefore in this "enclosure" that I have carried out my tests. Where a device resulted not accurate in YCbCr (or the YCbCr output setting is not present) I have tested its accuracy in RGB Full or Limited but with 16-235 videoscale. The display used is a TV Monitor Samsung SyncMaster P2370HD which, despite obsolescence, it has played its irrelevant task very well.
How I did the tests: DVDO AVLab TPG powered via USB from the laptop, TPG under analysis connected (one at a time) to the HDMI input of the DVDO with the HDMI output connected to the display, CalMAN 2018 Ethusiast R3 Beta Build 27, LightSpace CMS HTP 9.0.0.2448.
TPG for CalMAN:
Ted's Disc Media Files with third generation Apple TV
YCbCr 444 8 bit
MobileForge with Apple TV third generation
YCbCr 444 8 bit
RGB High
...and RGB Low
MobileForge with Apple TV 4K
YCbCr 444 8 bit
MobileForge with Amazon FireTV Stick
RGB Limited 8 bit
RPi PGENERATOR by Riccardo Biasiotto
RGB Full 8 bit
11.21.2018 UPDATE: As requested by [MENTION=8489745]ebr9999[/MENTION] I tested RGB limited output of RPi PGenerator. As previously stated, the not so bad results still prove that RPi is bit accurate only with RGB Full output with LightSpace or CalMAN videoscale set to 16-235.
Here it is:
END OF 11.21.2018 UPDATE
06.24.2019 UPDATE: As requested by [MENTION=8489745]ebr9999[/MENTION] I tested RPi PGenerator YCbCr limited output, videoscale set to 0-255. Again, as stated above, the results still prove that RPi is completely bit accurate only with RGB Full output with LightSpace or CalMAN videoscale set to 16-235. But, what is better for you, it depends on your TV RGB ---> YCbCr rounding errors.
Here it is:
END OF 06.24.2019 UPDATE
Dell Inspiron 17-7779 with CalMAN built-in TPG
RGB 8 bit
Ted's bluray disc with Panasonic DMP-BDT260
YcbCr 444 8 bit
Conclusions: As shown by the graphs, at the moment, MobileForge is not bit accurate with every device even if the output of the latter is (as in the case of Apple TV third generation, as proven by the test with the media files of Ted's disk). MAYBE wrong triplets are sent. With Apple TV 4K the situation seems to improve a bit but it's just an illusion. The Apple TV 4K, in fact, with output YCbCr and RGB seems to process the image continuously returning fluctuating values with digital errors ranging from a theoretical minimum of 1 to a theoretical maximum of 6 per triplet (as shown in the chart in which Ted patterns are used). For this reason, even if I reported the minimum values (by looking for them pixel by pixel) I did not calculate the relative dE for a simple reason: Apple TV 4K is not effectively usable as a TPG. That's why I will not proceed to test it further.
11.13.2018 UPDATE: Obviously, if ATV 4K is the only TPGable device you own you have no choice than use it. My only recommendation is to NOT measure the patches produced by it in contact mode, the more patch area you measure the more the digital errors will be reduced... forewarned is forearmed.
Since there are still some doubts about ATV 4K behavior with YCbCr/RGB high output, I filmed two videos, both displaying Ted's Color Checker Classic Media file via Infuse 5 Pro App with:
1) ATV 4K at 1080p 24 Hz YCbCr output
2) ATV 4K at 1080p 24 Hz RGB high output
NOTE: Except Rpi, I tested all devices with Ted's Color Checker Classic media files (or BD for my BD player) and all of them showed solid triplets except ATV 4K.
A very nice request to analyze CalMAN built-in with my laptop HDMI set to YCbCr has been made by baller02, I didn't think about it when I ran all the tests so I've analyzed Calman built-in TPG only with my laptop HDMI set to RGB and it turned out to be bit accurate. So prior to analyze CalMAN internal TPG in YCBCr, I have to do it with Ted's patterns and... Read this post for more.
END OF 11.13.2018 UPDATE
The Raspberry Pi (RPi) PGENERATOR by Riccardo Biasiotto is bit accurate and very stable (in RGB Full, not in YCbCr). We're talking about a DIY solution made by a NON PROFESSIONAL. Really outstanding, especially because it is 360° configurable, it connects via Ethernet, Wi-Fi, Bluetooth and it works with every calibration software I know!
Surprisingly, the HDMI output of my laptop resulted bit accurate!!!
TPG for LightSpace
LightSpace Connect with Apple TV third generation
YCbCr 444 8 bit
LightSpace Connect with Amazon FireTV Stick
RGB Limited 8 bit
RPi PGENERATOR by Riccardo Biasiotto
RGB Full 8 bit
Conclusions: Apple TV 4K has been excluded for the reasons mentioned above. All TPG solutions for LightSpace are bit accurate. The choice here could fall on the third generation Apple TV which is also perfect in YCbCr. On the same level, the RPi PGENERATOR by Riccardo Biasiotto.
Final Conclusions and tips: For those who only use CalMAN or even LightSpace, the only affordable and accurate bit solution is the RPi PGENERATOR by Riccardo Biasiotto.
For those who only use LightSpace and own a third-generation Apple TV, they can download and purchase LightSpace Connect but it is strongly suggested to use it connected via an Ethernet cable. Wi-Fi, during the generation of a color cube, could disconnect making you lose happy hours of real life. The RPi is also highly recommended with a super stable wireless Bluetooth connection.
Please, let me know your opinions, suggestions and (why not?!) your own tests!