Commit Graph

1022 Commits

Author SHA1 Message Date
Moritz Bunkus
88464c2f72 tests: add case for c7422daa4d 2017-06-24 10:58:59 +02:00
Moritz Bunkus
867ea96d04 mkvmerge: calculate NUMBER_OF_BYTES before content encoding is applied
Fixes #2022.
2017-06-24 10:56:47 +02:00
Moritz Bunkus
bc010d4c62 MPEG PS: probe for more (E-)AC-3 data if detection fails
Other track types such as DTS will already fetch more PS packets from
the stream if detection fails on the first packet. The same logic is
now applied to (E-)AC-3 tracks: as long as the track parameters cannot
be determined and the probe range hasn't been exceeded, fetch more
data from the stream and retry detection.

This enables track detection even if the first PS packets contain too
little (E-)AC-3 data.

Fixes #2016.
2017-06-24 09:25:21 +02:00
Moritz Bunkus
0bf1e14100 identification: output stream_id & sub_stream_id as integers; drop ts_pid 2017-06-22 20:04:40 +02:00
Moritz Bunkus
7915e67086 MPEG PS: ignore garbage at start during file type detection
Fixes #2008.
2017-06-19 18:59:19 +02:00
Moritz Bunkus
e6fcdfe093 MP4 reader: determine MPEG-1/2 version from ESDS if present
Fixes #1995.
2017-06-18 15:07:34 +02:00
Moritz Bunkus
401126db20 MPEG TS: parse SDT, output program info as container properties in identification result
SDT = service description table

The information output is a list of three-element maps:

• the program number
• the service provider's name (think TV station name, e.g. "ARD")
• the service's name (think TV channel name, e.g. "arte HD")

The program number corresponds to the track property `program_number`.

See #1990 for the future use case: presenting this information in the
GUI.
2017-05-25 12:14:22 +02:00
Moritz Bunkus
1fbada529f MPEG TS: parse all PMTs in streams with multiple programs properly
Earlier versions of mkvmerge used to detect all tracks in MPEG
transport streams with multiple programs, even though the code wasn't
really implemented & tested for that. However, some tracks (usually
those from the second or a later program) were broken: they might not
contain any data, or only invalid data.

On top of that mkvmerge v12.0.0 contains a fix for #1980 where a track
isn't part of a PMT at all. An unintentional consequence of that fix
was that mkvmerge no longer detected all of the tracks in
multi-program streams. The reason is that in order to detect tracks
not mentioned in a PMT mkvmerge has to do detection by content in the
PES packets. That's only implemented for AAC at the moment. All other
tracks will be blacklisted as soon as they're found.

This wouldn't be a problem if all PMTs of all programs were always
located right at the start of the file with nothing in
between. Unfortunately many files contain track content between
PMTs. So that workflow was:

• mkvmerge finds first PMT, determines types for tracks listed in it
• mkvmerge now considers the PMT to be found
• Continuing scanning the file mkvmerge encounters content for tracks
  not listed in the first PMT, attempting type detection by content,
  failing for most and blacklisting their PIDs
• Next a second PMT is found, however, the PIDs listed in that PMT may
  have already been found and blacklisted before — therefor they won't
  be considered anymore

With this fix mkvmerge actively looks for the PMTs for all
programs. Detection by content is only attempted once all PMTs have
been located. That way all tracks will be detected again.

A side effect of either this patch or one of the other ones before is
that the track content is now OK. I don't know exactly why or which
commit actually fixed it.

Fixes parts of #1990.
2017-05-22 19:43:03 +02:00
Moritz Bunkus
6010233729 MPEG TS: include program number in identification output 2017-05-22 17:56:17 +02:00
Moritz Bunkus
c69f3e73d7 JSON identification: include transport stream program number, bump version to 7 2017-05-22 17:52:59 +02:00
Moritz Bunkus
2a76ce4539 MPEG TS: make track IDs continuous 0..num_tracks -1
All tracks that weren't probed successfully (and which wouldn't have
been reported by the identification) will not be assigned an ID
anymore.
2017-05-21 18:43:22 +02:00
Moritz Bunkus
4496f4c75c tests: intentional update due to translation updates 2017-05-18 09:01:12 +02:00
Moritz Bunkus
96191d403c MP4 reader: support nclx colour type in colr atom
An enhancement to #1804.
2017-05-14 20:43:22 +02:00
Moritz Bunkus
548be05507 MP4 reader: find colr atom even if it's not first stsd atom extension
Another fix for the functionality introduced in #1804.
2017-05-14 20:42:59 +02:00
Moritz Bunkus
72a100aa11 mkvmerge: cue sheet parser: set segment title from cue sheet TITLE entry
Implements #1977.
2017-05-14 18:24:52 +02:00
Moritz Bunkus
8b7a81f9da MPEG TS: detect AAC & AC-3 tracks not listed in the PMT
If a packet is encountered for a PID that's not listed in the PMT,
mkvmerge will now attempt to determine its type by looking at the
first couple of bytes. Only checks for AAC (ADTS only, not LOAS) and
AC-3 are implemented.

Fixes #1980.
2017-05-11 22:25:20 +02:00
Moritz Bunkus
e1b27818e9 MPEG TS reader: require more consecutive packets, ignore h.264/h.265 start codes
In an earlier commit I introduced a workaround for h.264/h.265 files
being mis-detected as MPEG transport streams. That workaround was used
when the first bytes in a file were a valid h.264/h.265 start code.

Unfortunately this prevents the detection of valid MPEG transport
streams if they do indeed contain such a start code.

The fix is to remove the aforementioned workaround. Instead mkvmerge
now requires 333 KB of consecutive MPEG transport stream headers
inside the first 1 MB of the file, which amounts to ~1680 consecutive
headers. This reliably prevents the mis-detection as h.264/h.265 and
still allows for detection of real transport streams even if they
start with a h.264/h.265 start code.
2017-05-10 16:30:57 +02:00
Moritz Bunkus
c2cc3a788a MPEG TS reader: force AAC framer's multiplex mode to the detected mode
During file type detection the MPEG TS reader uses the AAC parser to
detect the multiplex mode. Later on it creates the AAC framer which in
turn contains its own instance of an AAC parser. This new instance
does its own multiplex mode detection.

For LOAS/LATM the detection can only succeed if the program mux
configuration is found. If it is not part of the first first PES
packet, then the framer's instance may get the detection wrong: it
does find LOAS/LATM headers, but as the program mux configuration
hasn't been parsed yet it'll continue detection and often happen over
ADTS headers instead.

The second detection is not only harmful, it's also superfluous as the
result is already known to the upper layer (the MPEG TS
reader). Therefore pass that information through from the reader via
the framer to the framer's parser.

Fixes #1957.
2017-04-24 17:03:24 +02:00
Moritz Bunkus
e5a9a8e7f7 HEVC parser: don't parse bitstream restriction twice if timing info is present in VUI
In order to fix #1924, I added bitstream restriction handling code in
the VUI parser in commit 2a385ab1ec.
Unfortunately I didn't realize that the code was present but in the
wrong place. It was only called if timing information was present,
too.

The result of commit 2a385ab1ec was that
the bitstream restriction was now handled twice if timing information
was present.

The superfluous and wrongfully-placed copy has been removed. This
fixes #1924 properly. It also fixes #1958.
2017-04-23 19:17:37 +02:00
Moritz Bunkus
cd742ed987 MP4 reader: fixes for the COLR atom support
• don't emit warnings or errors on failures, just don't parse the data
• fix structure size comparison (sizeof(structure), not sizeof(integer
  variable))
• update test suite checksums
• update AUTHORS
2017-04-22 17:00:51 +02:00
Moritz Bunkus
c30d3be97e AVC parser: remove all trailing zero bytes from NALUs
Fixes the AVC-specific part of #1952.
2017-04-22 16:37:28 +02:00
Moritz Bunkus
c008f20864 tests: cosmetics (alignment) 2017-04-22 15:19:14 +02:00
Moritz Bunkus
ebcda7f4eb version_number_t: add support for fewer than three digit groups 2017-04-22 15:17:53 +02:00
Moritz Bunkus
71094d135f tests: add unit test for version number parsing & formatting 2017-04-22 15:01:48 +02:00
Moritz Bunkus
23adff4c20 tests: update Korean language test due to c746c47cdc 2017-04-21 22:35:44 +02:00
Moritz Bunkus
5e48f483b0 AVC parser: use 25 FPS instead of bogus timing info values from SPS
Fixes #1946.
2017-04-17 20:32:45 +02:00
Moritz Bunkus
21eb1ecd5b mkvmerge: don't try to relocate written data when discarding
relocate_written_data is called in the following situation:

* track headers need to be re-written
* at least one frame has been written already
* the space left right after the track headers does not suffice to
  expand the track headers

In such a case all frames that have been written already will be
moved.

However, in certain split modes then current file may actually be a
null I/O, meaning that the current output is discarded. A null I/O
object doesn't return anything when reading for it, causing an endless
loop in the relocation code which calls `read` as often as needed
until everything's been read — which can never happen with a null I/O
object.

However, it makes no sense to try to actually read the data in such a
case, as it will be discarded anyway. Therefore just avoid trying to
read the data in the first place.

Fixes #1944.
2017-04-16 07:35:25 +02:00
Moritz Bunkus
e2a10ac071 FLAC reader: handle "picture" metadata blocks as attachments
Implements #1942.
2017-04-15 18:48:03 +02:00
Moritz Bunkus
243b70a03f tests: use "--engage no_variable_data" during identification 2017-04-15 18:46:43 +02:00
Moritz Bunkus
5fbcd0aa67 MPEG TS reader: require five consecutive, identical AAC headers for track type detection
Using only a single one may lead to false positives and consequently
to wrong track parameters, especially if the file was cut at an
arbitrary position.

Fixes the audio-related part of #1938.
2017-04-12 22:57:22 +02:00
Moritz Bunkus
44de7c3885 HEVC parser: don't store superfluous bytes at the end of the HEVCC
The old calculation method assumed that all picture set arrays are
always present in the HEVCC. This is not the case: arrays without
picture sets should not be written. Therefore their fixed size
overhead must not be added to the expected list size.

In order not to have to calculate the size in advance, the code has
been changed to write to an auto-resizing instance of mm_mem_io_c.

This is another fix for the video-related part of #1938.
2017-04-12 22:55:08 +02:00
Moritz Bunkus
c76e83ff83 HEVC parser: fix calculation of number of parameter set arrays in HEVCC
The number of parameter set arrays is not the sum of the number of
VPS, SPS, PPS and SEI NALUs, but the number of different types. For
example, if there's one VPS, one SPS, two PPS and no SEI NALUs, the
number of parameter set arrays must be three and not four.

Fixes the video-related part of #1938.
2017-04-12 18:34:57 +02:00
Moritz Bunkus
955eae59bf Matroska reader: treat invalid language elements as "und"
See #1929.
2017-04-04 19:18:43 +02:00
Moritz Bunkus
43f6c799db AVC/h.264 (framed) output: honor forced default duration signaling interlaced content
Internally the default duration given on the command line is stored as
the duration of a progressive frames. Additionally the framed
AVC/h.264 output module doesn't actually check whether or not the
current block contains a frame or a field. This combination leads to
the situation that specifying a default duration that signals
interlacing (e.g. 50i) results in an actual default duration of 40ms,
that of a progressive frame.

This change passes the information provided by the user about frame
vs. fields from the command line through to the output module so that
it can react accordingly.

Fixes #1916.
2017-03-31 16:50:42 +02:00
Moritz Bunkus
2a385ab1ec HEVC parser: copy bitstream restriction info in VUI parameters
The VUI parameter copy method was simply missing the code for copying
the "bitstream_restriction_flag" and all of its dependant
fields (see ITU-T "H.265 12/2016" annex E.2.1).

Fixes #1924.
2017-03-30 20:10:57 +02:00
Moritz Bunkus
5e49abc525 tests: fix version number test for v10.0.0
It's the first version number where a component is more than one digit
long.
2017-03-25 15:17:09 +01:00
Moritz Bunkus
954caf782a mkvextract: SSA/ASS: follow given "Format" line
Before this commit mkvextract used to assume that the "Format" line
always contained all known fields. For files where this wasn't the
case the text was then empty as mkvextract was assuming that there are
more fields present than there actually were.

This commit changes mkvextract to simply follow the given field order
from an existing "Format" line.

Fixes #1913.
2017-03-20 14:11:26 +01:00
Moritz Bunkus
ccf3f6b2c6 tests: move "ssa-ass" directory to "subtitles" sub-directory 2017-03-20 13:28:21 +01:00
Moritz Bunkus
639407f580 HEVC parser: detect and ignore invalid default display window VUI parameters
Certain files seem to lack the "default display window" data (well,
the flag that signals its presence/absence). Therefore a
standards-compliant parser would try to read that flag, but the data
read would belong to another flag (the "vui_timing_info_present_flag").

ffmpeg has a heuristic in place for detecting such invalid default
display window parameters. This commit implements the same heuristic
in mkvmerge.

Fixes #1907.
2017-03-19 18:57:17 +01:00
Moritz Bunkus
0acdc0e51c AVC parser: drop all frames before the first key frame
See #1908.
2017-03-18 18:09:11 +01:00
Moritz Bunkus
a19459a7fa AVC parser: only use last provided timestamp for a given stream position
In certain cases there are several timestamps from the source
container queued up for a given position of the NALU in the stream. In
such cases using the first available timestamp will result in
audio/video desync. Instead the last timestamp whose stream position
is smaller than or equal to the NALU's stream position should be used.

Fixes #1908.
2017-03-18 17:22:19 +01:00
Moritz Bunkus
e5b5885f37 mkvmerge: Matroska: handle timestamps < 0 properly
mkvmerge will now shift all timestamps in a file up so that no
timestamp read from the file is smaller than zero. Before this change
it was up to the output module to cope with timestamps < 0, which most
simply couldn't, and the result was audio/video desynchronization.
2017-03-08 16:42:14 +01:00
Moritz Bunkus
87d9363812 MP4 reader: complete rewrite of the composition time & edit list handling
The handling for edit lists ('elst' atoms) and composition timestamp
offsets ('ctts' atoms) were not working well together causing offsets
to be applied to certain track types in certain situations. This led
to offsets between tracks, e.g. when no edit lists were in play but
CTTS atoms were used.

Fixes #1889.
2017-02-24 14:15:54 +01:00
Moritz Bunkus
daab575faf MP4 reader: always observe DTS-to-CTS offset, not just for h.264/h.265 2017-02-18 21:55:28 +01:00
Moritz Bunkus
38478ea128 MP4 reader: adjust timestamps of all muxed tracks to be zero-based
Before only shifting from negative timestamps up to zero. However, if
all timestamps of all tracks were greater than zero, then they weren't
adjusted down.
2017-02-18 20:29:57 +01:00
Moritz Bunkus
e6149128a4 math: add function for converting unsigned integers to signed ones
This should be used for reading signed integers from files where
they're stored in 2's complements (as all modern processors use). In
cases where the unsigned integer is bigger than the maximum signed
integer, static_cast<>ing is an implementation-defined behavior.

The code was adopted from the following answer on Stack Overflow where
they discuss an implementation that avoids such implementation-defined
behavior: https://stackoverflow.com/a/13208789/507077
2017-02-18 19:23:13 +01:00
Moritz Bunkus
5d2f7eb039 mkvmerge: probe AC-3/MP3/AAC at start of file before in middle of file
Fixes certain misdetection scenarios.
2017-02-18 12:34:10 +01:00
Moritz Bunkus
c52e85e94c Matroska reader: DVB subtitles: add subtitling type byte if missing in CodecPrivate 2017-02-17 18:39:30 +01:00
Moritz Bunkus
61f83424af refactor item_selector_c to use boost::optinal, make demuxing_requested const 2017-02-14 10:42:08 +01:00
Moritz Bunkus
568affaaf4 MP4 reader: fix refactoring mistake: forgotten "return" after "if …"
See #1883, where this problem first came to light, even though 1883
itself was about another problem.
2017-02-11 17:30:39 +01:00