Directional detection of fast neutrons emitted by special nuclear materials can be performed with a time projection chamber. This device permits particle identification and full three-dimensional reconstruction of charged-particle tracks produced by interaction of fast neutrons in the chamber active volume. Single-recoil-proton reconstruction allows rapid pointing, while the reconstruction of two recoil protons produced by a single incident neutron event can enable a measurement with very high angular resolution. Kinematic reconstruction algorithms for both of these cases are presented and their performance assessed using data generated by a simple Monte Carlo simulation and experimental data where those exist. The simulation data are also used to estimate the relative efficiency of both neutron imaging modalities as a function of the volume and pressure of the time projection chamber.