TODO.txt

The Big Picture
===============

- Lots of documentation
- Uniform time representation
- High-level APIs


Encoding/Decoding Without Streams
=================================

- See: http://ffmpeg.org/doxygen/trunk/decoding_encoding_8c-example.html
- Some codecs need width/height/pix_fmt(??) before decoding, because that
  isn't in the bitstream.
- Some codecs need packets that contain exactly a frame, but some can deal with
  streams.
- It feels like we might need a mutable Packet
  Have the packet retain it's original data/size, and appear to deplete as it
  is consumed.
- Parsers are interesting: https://www.ffmpeg.org/doxygen/trunk/structAVCodecParser.html


Reference Counting
==================

- Switch everything to av_*_unref ??
    - av_frame_unref
    - av_packet_unret


Filters
=======

- Expose options on filter classes.

- Graph.add_buffer(template_frame)
- Graph.add_abuffer(template_frame)
- Graph.add_sink(template_frame)
- Graph.add_asink(template_frame)

- Graph.parse()
    - create FilterContext objects for every newly created filter
    - return inputs/outputs

- FilterContext.push(frame, input=None) -> If this is not a (a)buffer, check the given input
  (or the only input if there is one). If nothing is linked to that input, create a buffer there.
  Then push it in.
- FilterContext.pull(frame, output=None) -> Similar semantics to push above.

- Graph.push(frame) -> push the frame into the single input, if it exists.
- Graph.pull(frame) -> pull the frame from the single output, if it exists.

- FilterContext.send_command(cmd, arg)
- FilterContext.{inputs,outputs} -> tuple of FilterPad


Uniform time
============

class Time():

  # Tracks the last value set.
  cdef bint is_rational

  cdef AVRational base
  cdef uint64_t i
  cdef double d

  def __int__(self):
    # return i or transform d

  def __float__(self):
    # return d or transform i
  
  def rebase(self, new_base):
    # Replace the base, and rescale i (if important)


Encoding
========

- Stream.encode should return a list of packets. Otherwise, resampling
  can easily result in a massive buffer in a fifo or codec

- Make everything mutable. Later, we can introduce a PseudoMutable base class
  for everything that has a .assert_mutable() method and public is_read_only


Documentation
=============

- Document/test the entire API (in the most casual way).
- Relink everything back to the underlying library calls, so that cross
  referencing can be done.
- Automatically scan the Cython source to establish when it uses
  library functions? This may result in far too many links to be useful.


Build
=====

- Include build discovery info into the module, retrievable by `python -m av --version`


Time
====

- What are the various rates and scales?

  see: http://ffmpeg.org/pipermail/libav-user/2011-December/001041.html
  see: http://stackoverflow.com/questions/12234949/ffmpeg-time-unit-explanation-and-av-seek-frame-method/16739755#16739755

  - Stream.guessed_rate or AVStream.r_frame_rate:
    FFmpeg's guess as to the frame rate. Often 24/1.

  - Stream.average_rate or AVStream.avg_frame_rate:
    Average frame rate that has been inspected in the file. This is what VLC
    reports in the media information. Often near 24/1.

    It may calculate this from 20 frames; see: http://ffmpeg.org/doxygen/trunk/libavformat_2utils_8c_source.html#l02768

  - Stream.time_base or AVStream.time_base:
    Duration of one time unit for AVStream times.

  - Stream.rate or CodecContext.time_base:
      The length of a single frame as reported by the file. Sometimes 1/24,
      sometimes really really wrong.

  possibilities for discrepencies:
    - .mov supports a fixed frame rate
    - .mp4 does not support a fixed frame rate?
    - see: http://ffmpeg.org/doxygen/trunk/libavformat_2utils_8c_source.html#l02647


High-Level API
==============

- Container.demux(video=True)
- Container.demux(video=0, audio=0) first video and audio
- Container.decode(*args_for_demux, **kwargs_for_demux) -> straight to frames
- Stream.mux(frame) -> encode, flush, and mux
- Container.__del__ -> flush the streams and close
√ Container.add_stream(template=stream)


Unsorted
========

- VideoFrame.from_file(file_or_path); opens the file and decodes the first frame)

- InternalObject class for stopping construction:

    __cinit__(self, sentinel):
        assert sentinel
    
    @classmethod
    cdef alloc(cls, ...):
        self = cls(sentinel)
        self._init(*args, **kwargs)
    
- isdeferredinstance loads the module via sys.modules, then getattr on it, and
  finally do an isinstance.
  
  isdeferredinstance(x, ('PIL.Image', 'Image'))

- Expose CodecContext as stream.context, and move properties to it.

- Rename Buffer and ByteSource to represent they are opposite directions of
  the same operation.

- `atexit.register` something to clean up FFmpeg threads

- Directly use the Numpy C-API
  - see: http://docs.scipy.org/doc/numpy/reference/c-api.array.html
  - see: https://github.com/cython/cython/wiki/tutorials-numpy#c-api-initalization

- Move the logic regard initializating a VideoReformatter into the object
  itself instead of the VideoFrame.reformat method.

- Protocol for streams to pull packets from iterators. Then secondary streams
  get easier: container.add_stream('audio_codec', source=av.open('music.aiff').iter_packets(only_primary=True))

- Create Codec Descriptor

- Make av.utils.SmartPointer to wrap around a pointer? All this would do is
  hold a reference to said pointer and make sure it isn't garbage collected.

runtime_library_dirs

- Look at how stream alloc works; can we manually do this and register the
  stream on a format context later?

- Various AudioLayout/AudioFormat/VideoFormat attributes should be writable.
    - is_mutable flags on various objects (including formats, layouts, contexts,
      streams, etc.) could guard the __set__ methods of properties.

- Should methods be: to_bytes, as_bytes, tobytes, asbytes??

- Plane.array_format could be a format string for array.array
- Plane.update_from_array(array.array)
    - If it were HD RGB then it would be 1920 * 1080 * 3 long.
- Plane.update_from_ndarray(numpy.ndarray)
    - If it were HD RGB then it would have shape (1080, 1920, 3)
- Plane.update(input_)
    And then it tries if it is a buffer, memoryview, bytes, etc..

- Stream.encode(...) -> list of packets, automatically checking buffer)
- Context.mux(...) -> take a single packet, or an iterator of them:
        context.mux(stream.encode(frame))

- Context.add_stream(codec_name, frame_rate) -> Context.streams.append(Stream(codec_name, frame_rate))?

- TestCase.rms_diff(one, two) -> Root-mean-square diff
- try to wrap API of testsrc filters

- FFmpeg tutorial: http://dranger.com/ffmpeg/
	- also has function reference: http://dranger.com/ffmpeg/functions.html
	- updated tutorial code: https://github.com/chelyaev/ffmpeg-tutorial

- Even out more of the differences:
    - See README of https://github.com/chelyaev/ffmpeg-tutorial

- Replicate av_frame_get_best_effort_timestamp

    http://ffmpeg.org/pipermail/ffmpeg-devel/2011-February/104327.html
    http://pastebin.com/Aq8eDZw3/
    http://web.archiveorange.com/archive/v/yR2T4bybpYnYCUXmzAI5