Techwatch: Emerging Video Codecs (AV1)

[Rodney]

This topic was previously titled: Thor - a new (open) video codec championed by Cisco

 

 

 

For those with an interest in such things...

 

https://tools.ietf.org/html/draft-fuldseth-netvc-thor-00

 

The primary impetus for bringing it into existence appears to be to ensure the codec is free from patents.

The linked document goes into detail of how the codec processes data.

[robcat2075]

Attempt to combat creeping Silicon Valley hipster irony...

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].

[NancyGormezano]

Attempt to combat creeping Silicon Valley hipster irony...

 

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",

"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this

document are to be interpreted as described in RFC 2119 [RFC2119].

 

hoo-hooooo!

 

This, young man, is exactly why I retired from the exciting world of government contracting at an early age.

 

Government spec document writing requirements in engineering would bring me to soft sobs and eventually loud wailing in my regulation sized, windowless cubicle, wondering "what have I done wrong in a previous life, to suffer this anal-retentive drivel?"

 

Soul-sucking conformity in document writing has been around since the "dust ages".

[John Bigboote]

Gobleddy-gook!

 

I wannsa see some 'fr-instances'... like 'here is a 100mb uncompressed video file...here it is compressed with H264, Mpeg, Pro-Res and their file sizes and respective qualities... now HEEEERE, is our NEW .thor compression... LOOK how good the quality and MY how LOW the file size is!!!'

 

Until they do that...

[robcat2075]

Gobleddy-gook!

 

I wannsa see some 'fr-instances'... like 'here is a 100mb uncompressed video file...here it is compressed with H264, Mpeg, Pro-Res and their file sizes and respective qualities... now HEEEERE, is our NEW .thor compression... LOOK how good the quality and MY how LOW the file size is!!!'

 

Until they do that...

 

And this was the "high-level" explanation. Imagine what the low-level details are like.

[Rodney]

I wannsa see some 'fr-instances'... like 'here is a 100mb uncompressed video file...here it is compressed with H264, Mpeg, Pro-Res and their file sizes and respective qualities... now HEEEERE, is our NEW .thor compression... LOOK how good the quality and MY how LOW the file size is!!!'

 

 

I believe the point might be getting missed...

 

One could hope a new solution/technology is superior but again... that's not the (presumed) underlying point of the Thor codec.

As a caviet I wouldn't suggest that an additional goal wouldn't also be to improve the technology (as I'm sure anyone solving the problem would desire that too) but the PRIMARY POINT of the Thor codec appears to be achieving a PATENT FREE codec that everyone can use without free of legal ramifications (i.e. no one can sue you because you've used the technology in your code).**

 

This should better explain the reason... or necessity if you prefer... to LEGALLY define every minute detail of the codec.

This also might explain why there are likely more legal consultants on the Thor codec project than programmers.

 

To the end user that expects usage without concern for the patents involved this wont resonate very deeply but for those concerned that the code they are putting into their programs could have serious ramifications it might ease their minds a bit.

 

And the technicalities in this realm will rule the day because those are the constraints that must be worked.

 

A little more explanation (although almost as technical) can be found here:

 

http://recordings.conf.meetecho.com/Playout/watch.jsp?recording=IETF93_NETVC_II&chapter=chapter_1

 

(Non-technical types should probably avoid this as it won't serve much of a useful purpose)

I believe the entire part of the presentation that relates directly to Thor codec ends about 34 minutes into the presentation although following presentations do relate to video/codecs.

 

Note: It takes a little while for the show to get started.

For Matt: This does get into technicalaiies and comparisons of other comparable codecs (VP9/H265) and at about 25 minutes into the video there is a compression comparison chart.

I'm not suggesting you actually give this a look... only that the information you suggest would be most useful is starting to appear and the codec is developed.

 

**This doesn't mean Cisco doesn't have other longer term goals that make development/support of the patent free video codec advantageous to them financially. I assume there they have carefully calculated their ROI.

 

Added: It's interesting to note who the questioners are in the audience and what they offer to bring to the project.

Edited August 12, 2015 by Rodney
Minor edits for clarification/refinement

[Rodney]

Others are joining the effort to create patent free and open media formats:

 

http://aomedia.org/press-release/alliance-to-deliver-next-generation-open-media-formats/

 

From the Alliance for Open Media site:

 

Sept. 1, 2015 – Seven leading Internet companies today announced formation of the Alliance for Open Media – an open-source project that will develop next-generation media formats, codecs and technologies in the public interest. The Alliance’s founding members are Amazon, Cisco, Google, Intel Corporation, Microsoft, Mozilla and Netflix. The new Alliance is committing its collective technology and expertise to meet growing Internet demand for top-quality video, audio, imagery and streaming across devices of all kinds and for users worldwide.

The Alliance’s initial focus is to deliver a next-generation video format that is:

• Interoperable and open;
• Optimized for the web;
• Scalable to any modern device at any bandwidth;
• Designed with a low computational footprint and optimized for hardware;
• Capable of consistent, highest-quality, real-time video delivery; and
• Flexible for both commercial and non-commercial content, including user-generated content.

This initial project will create a new, open royalty-free video codec specification based on the contributions of members, along with binding specifications for media format, content encryption and adaptive streaming, thereby creating opportunities for next-generation media experiences.

 

From Techcrunch:

The group plans to publish its code under the Apache 2.0 license and it will operate under W3C patent rules, meaning the members will waive royalties from the codec implementations and their patents on the codec itself.

 

[Rodney]

It looks like the Alliance for Open Media's first foray into video (AV1 codec) is making some progress.

 

 

“We are very excited to be joining the Alliance for Open Media and to be the first company to integrate the AOMedia Video codec, AV1, into our products supporting both live streaming and video-on-demand,” said Stefan Lederer, CEO of Bitmovin. “This alliance exemplifies our focus on being a first mover in the video streaming domain and we look forward to working with the other members to deliver a next-generation video format.”

The availability of AOMedia Video as an open source project is an important element in fulfilling the organization’s promise to deliver a next-generation video format that is:

· Interoperable and open;

· Optimized for internet delivery;

· Scalable to any modern device at any bandwidth;

· Designed with a low computational footprint and optimized for hardware;

· Capable of consistent, highest-quality, real-time video delivery; and

· Flexible for both commercial and non-commercial content, including user-generated content.

"We’re pleased to welcome Bitmovin to the Alliance for Open Media, reflecting the importance of video encoding and player providers’ engagement to achieve broad industry adoption," said Gabe Frost, Executive Director, the Alliance for Open Media. "We look forward to Bitmovin's contributions to this significant initiative, which will open the door to a new era of openness and interoperability for internet video."

 

As for the Thor codec (topic title), while still presumably proceeding, it looks to me like the tech involved has mostly been gobbled up for use in other product.

 

 

Added: The early roll out of AV1 likely won't effect the low hanging fruit of the non-corporate world as there is money to be made and recouped. I only highlight the emergence of new codecs/standards because they appear to be headed (eventually) toward us.

[Rodney]

There are some interesting components of the AV1 codec that catch my eye.

 

It would appear that with Google as part of the Alliance for Open Media that they will likely deprecate the VP9 format in favor of AV1 (and eventually AV2 etc).

 

Here's a bit about that:

The new video coding format AOMedia Video 1 (AV1) is meant to replace Google’s VP9 and compete with HEVC/H265 from MPEG. The Alliance is targeting an improvement of about 50% over VP9/HEVC with only reasonable increases in encoding and playback complexity.

 

 

Regarding comparison to HEVC:

When comparing AV1 with HEVC, probably the biggest competitive advantage of AV1 will be that it is royalty-free, especially if we look at the still very uncertain royalty situation with HEVC. Currently there are two patent pools with MPEG LA and MPEG Advance, plus some unknown HEVC IP owners who have not joined a pool yet. In the end, nobody will know how much you will need to pay in royalties for HEVC. This situation is obviously not satisfactory for the industry and especially, encoding, distribution, content and hardware companies.

 

 

 

More importantly is what the codec might bring to use but first a little background:

The AV1 codec has its roots in the codebase of Google’s VP9/VP10 codec with an additional 77 experimental coding tools that have been added and are under consideration. Out of that 77 experimental coding tools, only 8 are currently enabled by default (adapt_scan, ref_mv, filter_7bit, reference_buffer, delte_q, tile_groups, rect_tx, cdef), but the performance of the codec is already appealing. The final goal is to get as many promising coding tools into the final version of the codec and afterwards freeze the bitstream specification.

 

 

The most promising of the coding tools (experiments) are said to be:

 

Directional Deringing

It is an effective algorithm for removing ringing artifacts from a coded frame. It plugs in right at the end of the decoding process, so it is easy to integrate. Blocks are searched for an overall direction that is taken into account when applying a conditional replacement filter (CRF) to reduce the risk of blurring and only take obvious ringing patterns into account. It is currently enabled by default.

While not necessarily related, I like the thought of 'predictive' rendering and when directions (in timespace) are analyzed it can inform and reduce processing time.

PVQ (Perceptual Vector Quantization)

This experiment was originally developed for the Daala codec and has the potential to bring a lot of gains, however, it is also quite difficult to integrate into AV1 because PVQ interacts with many other parts of a codec. Compared to the usual scalar quantization, PVQ offers a lot more flexibility to control quantization. It makes techniques like Chroma from Luma or Activity Masking easier. Activity Masking is trying to provide better resolution in low contrast areas. This can be achieved by varying the codebook which is possible with PVQ.

While there is much to be found herein what comes immediately to mind is 'seam carving' for video. For still imagery this technology is often referred to as 'healing/healing brush' because that is the effect accomplished in programs such as Photoshop.

Chroma from Luma (CfL)

CfL is based on a rather simple idea: Take advantage of the fact that edges in the chroma plane are usually well correlated with those in the luma plane. As CfL works entirely in the frequency domain, it can be easily implemented using PVQ. Using PVQ, the chroma coefficients can be predicted from injected luma coefficients. It is a very promising tool as it is quite simple to compute and provides nice benefits with much cleaner colors.

 

Hey if it helps to identify objects in depth and separate them (ala automated masking) then I'm all for it.

All quotes source from: Link

[robcat2075]

If this codec can will make me look 20 years younger, I'm all for it.

[Rodney]

If this codec can will make me look 20 years younger, I'm all for it.

 

One down, billions to go.

[John Bigboote]

I am seeing people 'in the biz' turning to Jpeg2000 or even GoPro codecs lately, as the Animation Compression seems to have gone wonky on us.

[Rodney]

VideoLAN Joins the Alliance for Open Media

VideoLAN, a non-profit organization that develops and promotes open source multimedia solutions, has joined the Alliance for Open Media, a non-profit effort to develop a new open and royalty-free video codec: AV-1. VideoLAN will bring nearly 20 years of experience developing open source multimedia software to help build the next-generation video codec.

Since its creation, the VideoLAN project has worked to make multimedia simple for everyone, users and professionals. To achieve that goal, VideoLAN has developed several pieces of open source software, including the VLC media player, the x264 encoder and the libdvdcss library. The organization also supports and funds other open source projects, such as; FFmpeg, Libav, and Handbrake.

 

Another addition to the Alliance... and an important one at that.

If their work with Handbrake, VLC and FFmpeg are any indication of things to come then that bodes well for the Alliance for Open Media and the AV1 format/codec.

Full announcement (from VideoLAN): LINK

Aside: I haven't been watching version numbers closely but apparently after 13 years of development the creators of Handbrake recently released version 1.0.0. The latest version as of this writing is v1.0.7. For more information about Handbrake and for the only official download site: LINK

[Rodney]

More progress being made toward the pending release of the AV1 video format (targeted for 2018).

 

Although the standard itself is still not fully set, Firefox is the first to release a browser that can play AV1.

 

For the latest from Mozilla and Bitmovin:

https://hacks.mozilla.org/2017/11/dash-playback-of-av1-video/

 

From the article:

the industry is gearing for fast adoption of the new codec, which promises to be 25-35% more efficient than VP9 and H.265/HEVC.

 

For those that have the latest Firefox nightly installed a demo page can be found here:

 

https://demo.bitmovin.com/public/firefox/av1/

 

The demo film is 'Tears of Steel' which is a tip of the hat to open source, patent free and similar movements in the tech world.

[robcat2075]

Hmmm... the page says I need Firefox 57.0 and I have 57.0 but I still get no video.

 

EDIT: after I reloaded the page it gave me a link to down load the "Nightly".

 

I'd say they still have work to do. All I got was the first few seconds of video and then stuck frames even though the progress bar was far ahead of the play point.

[Rodney]

I'd say they still have work to do. All I got was the first few seconds of video and then stuck frames even though the progress bar was far ahead of the play point.

 

They do indeed. I'm sure it's a lot of work being out there on the cutting edge.

I'm a bit surprised it even works.

 

Most people who work with 'nightlies' are into bug testing and working out all the rough spots.

The bugs definitely come out at night.

 

Gotta love tech watches... lots of waiting and watching.

And by the time the thing arrives... most folks have already moved on.

[Rodney]

It seems more than a few people were taken by surprise when a few weeks ago Apple joined the Alliance for Open Media.

They were the big holdout.

 

CNET article (this has some info related to the AV1 format too):

 

https://www.cnet.com/news/apple-online-video-compression-av1/

 

 

The first article I saw:

 

https://bitmovin.com/apple-joins-av1-codec-consortium/?utm_source=bitmovin_newsletter&utm_medium=email

[Rodney]

The AV1 video format continues to move forward with support from all major browsers forthcoming and expected hardware to fully support the format by 2020.

 

Here's a recent article that fills in some additional information and that reports on some of the competing technology:

The article also captures some of posturing from those that expect to lose revenue due to the impending release of the patent free format.

 

https://www.cnet.com/news/netflix-youtube-streaming-video-is-about-to-get-a-lot-faster-av1-compression/