The Matrix Cookbook
TL;DR — Every matrix identity you need and none of the prose. The canonical answer to “most underrated PDF” for anyone who has done the derivations.
⬇ Download the archived copy — kept here so the document survives its source going dark. Explicitly free / author-released / CC.
72 pages of matrix identities, derivatives, decompositions and inverses, with no exposition at all.
It is a lookup table, and that is the design. Anyone deriving gradients by hand — which is to say anyone who learned machine learning before autodiff, and anyone debugging autodiff since — has a copy.
Free, maintained, and the single most-recommended reference document of its type, which is why it appears in a thread like this rather than in a textbook list.
Where this came from
72 pages. A copy is archived locally against link rot; the header links the original source.