This is preparation for making the GUI present the encoding detected
by e.g. a byte order mark to the user so that she doesn't have to (and
cannot) chose a subtitle character set herself. Implements mkvmerge's
part of #2053.
Also refactors SBR detection & handling. There were cases in which
reading AAC from MPEG TS resulted in 2-byte `CodecPrivate` with no
`OutputSamplingFrequency` set, even though the sampling frequency was
24.000 Hz and therefore an indicator for SBR.
The feature was implemented by removing all 0 bytes in before the next
start code (and all 0 bytes at the end of the buffer). The problem is
that a slice structure may very well end in 0 bytes. The only way to
determine the end of the slice structure with confidence is
implementing a parser for the whole slice structure.
The result of removing bytes belonging to the slice structure may or
may not end in visual artifacts upon decoding. Other results include
error message by the decoder (e.g. ffmpeg which reports errors such as
"slice mismatch" or "motion vectors not available").
I lack the time and motivation to implement a proper slice parser. As
the current behavior is dangerous and just plain wrong, I'm removing
the feature again. It was introduced in release 5.8.0 response to
issue #734, which will now remain not implemented.
Fixes#2045.
Part of an ongoing effort to replace the use of the term `timecode`
with `timestamp`. Timecodes have a very specific meaning in the
audio/video world, and it's not what MKVToolNix has been using the
term for.
Old pugixml versions didn't recode to UTF-8 themselves. Therefore
mkvmerge contains code to determine the encoding from the XML
declaration's `encoding` attribute and recoding the content to UTF-8
before passing it to pugixml. The old `encoding` attribute was left
untouched.
Newer pugixml versions do recode to UTF-8. In order to prevent
that (and therefore recoding twice), the XML declaration's `encoding`
attribute must be set to `UTF-8`.
Whenever a sequence parameter set or picture parameter set
changes (meaning an SPS with the same ID as an earlier SPS but with
different content is found), all frames queued for order & timestamp
calculation must be flushed. Otherwise frame order calculation will be
based on wrong values for some frames and on correct values for other
frames.
Fixes#2028.
Other track types such as DTS will already fetch more PS packets from
the stream if detection fails on the first packet. The same logic is
now applied to (E-)AC-3 tracks: as long as the track parameters cannot
be determined and the probe range hasn't been exceeded, fetch more
data from the stream and retry detection.
This enables track detection even if the first PS packets contain too
little (E-)AC-3 data.
Fixes#2016.
SDT = service description table
The information output is a list of three-element maps:
• the program number
• the service provider's name (think TV station name, e.g. "ARD")
• the service's name (think TV channel name, e.g. "arte HD")
The program number corresponds to the track property `program_number`.
See #1990 for the future use case: presenting this information in the
GUI.
Earlier versions of mkvmerge used to detect all tracks in MPEG
transport streams with multiple programs, even though the code wasn't
really implemented & tested for that. However, some tracks (usually
those from the second or a later program) were broken: they might not
contain any data, or only invalid data.
On top of that mkvmerge v12.0.0 contains a fix for #1980 where a track
isn't part of a PMT at all. An unintentional consequence of that fix
was that mkvmerge no longer detected all of the tracks in
multi-program streams. The reason is that in order to detect tracks
not mentioned in a PMT mkvmerge has to do detection by content in the
PES packets. That's only implemented for AAC at the moment. All other
tracks will be blacklisted as soon as they're found.
This wouldn't be a problem if all PMTs of all programs were always
located right at the start of the file with nothing in
between. Unfortunately many files contain track content between
PMTs. So that workflow was:
• mkvmerge finds first PMT, determines types for tracks listed in it
• mkvmerge now considers the PMT to be found
• Continuing scanning the file mkvmerge encounters content for tracks
not listed in the first PMT, attempting type detection by content,
failing for most and blacklisting their PIDs
• Next a second PMT is found, however, the PIDs listed in that PMT may
have already been found and blacklisted before — therefor they won't
be considered anymore
With this fix mkvmerge actively looks for the PMTs for all
programs. Detection by content is only attempted once all PMTs have
been located. That way all tracks will be detected again.
A side effect of either this patch or one of the other ones before is
that the track content is now OK. I don't know exactly why or which
commit actually fixed it.
Fixes parts of #1990.
If a packet is encountered for a PID that's not listed in the PMT,
mkvmerge will now attempt to determine its type by looking at the
first couple of bytes. Only checks for AAC (ADTS only, not LOAS) and
AC-3 are implemented.
Fixes#1980.
In an earlier commit I introduced a workaround for h.264/h.265 files
being mis-detected as MPEG transport streams. That workaround was used
when the first bytes in a file were a valid h.264/h.265 start code.
Unfortunately this prevents the detection of valid MPEG transport
streams if they do indeed contain such a start code.
The fix is to remove the aforementioned workaround. Instead mkvmerge
now requires 333 KB of consecutive MPEG transport stream headers
inside the first 1 MB of the file, which amounts to ~1680 consecutive
headers. This reliably prevents the mis-detection as h.264/h.265 and
still allows for detection of real transport streams even if they
start with a h.264/h.265 start code.
During file type detection the MPEG TS reader uses the AAC parser to
detect the multiplex mode. Later on it creates the AAC framer which in
turn contains its own instance of an AAC parser. This new instance
does its own multiplex mode detection.
For LOAS/LATM the detection can only succeed if the program mux
configuration is found. If it is not part of the first first PES
packet, then the framer's instance may get the detection wrong: it
does find LOAS/LATM headers, but as the program mux configuration
hasn't been parsed yet it'll continue detection and often happen over
ADTS headers instead.
The second detection is not only harmful, it's also superfluous as the
result is already known to the upper layer (the MPEG TS
reader). Therefore pass that information through from the reader via
the framer to the framer's parser.
Fixes#1957.
In order to fix#1924, I added bitstream restriction handling code in
the VUI parser in commit 2a385ab1ec.
Unfortunately I didn't realize that the code was present but in the
wrong place. It was only called if timing information was present,
too.
The result of commit 2a385ab1ec was that
the bitstream restriction was now handled twice if timing information
was present.
The superfluous and wrongfully-placed copy has been removed. This
fixes#1924 properly. It also fixes#1958.
• don't emit warnings or errors on failures, just don't parse the data
• fix structure size comparison (sizeof(structure), not sizeof(integer
variable))
• update test suite checksums
• update AUTHORS
relocate_written_data is called in the following situation:
* track headers need to be re-written
* at least one frame has been written already
* the space left right after the track headers does not suffice to
expand the track headers
In such a case all frames that have been written already will be
moved.
However, in certain split modes then current file may actually be a
null I/O, meaning that the current output is discarded. A null I/O
object doesn't return anything when reading for it, causing an endless
loop in the relocation code which calls `read` as often as needed
until everything's been read — which can never happen with a null I/O
object.
However, it makes no sense to try to actually read the data in such a
case, as it will be discarded anyway. Therefore just avoid trying to
read the data in the first place.
Fixes#1944.
Using only a single one may lead to false positives and consequently
to wrong track parameters, especially if the file was cut at an
arbitrary position.
Fixes the audio-related part of #1938.
The old calculation method assumed that all picture set arrays are
always present in the HEVCC. This is not the case: arrays without
picture sets should not be written. Therefore their fixed size
overhead must not be added to the expected list size.
In order not to have to calculate the size in advance, the code has
been changed to write to an auto-resizing instance of mm_mem_io_c.
This is another fix for the video-related part of #1938.
The number of parameter set arrays is not the sum of the number of
VPS, SPS, PPS and SEI NALUs, but the number of different types. For
example, if there's one VPS, one SPS, two PPS and no SEI NALUs, the
number of parameter set arrays must be three and not four.
Fixes the video-related part of #1938.
Internally the default duration given on the command line is stored as
the duration of a progressive frames. Additionally the framed
AVC/h.264 output module doesn't actually check whether or not the
current block contains a frame or a field. This combination leads to
the situation that specifying a default duration that signals
interlacing (e.g. 50i) results in an actual default duration of 40ms,
that of a progressive frame.
This change passes the information provided by the user about frame
vs. fields from the command line through to the output module so that
it can react accordingly.
Fixes#1916.
The VUI parameter copy method was simply missing the code for copying
the "bitstream_restriction_flag" and all of its dependant
fields (see ITU-T "H.265 12/2016" annex E.2.1).
Fixes#1924.
Before this commit mkvextract used to assume that the "Format" line
always contained all known fields. For files where this wasn't the
case the text was then empty as mkvextract was assuming that there are
more fields present than there actually were.
This commit changes mkvextract to simply follow the given field order
from an existing "Format" line.
Fixes#1913.
