We present an integrated magneto-photonic device for all-optical switching of non-volatile multi-bit spintronic memory. The bits are based on stand-alone magneto-tunnel junctions which are perpendicularly magnetized with all-optically switchable free layers, coupled onto photonic crystal nanobeam cavities on an indium phosphide based platform. This device enables switching of the magnetization state of the bits by locally increasing the power absorption of light at resonance with the cavity. We design an add/drop network of cavities to grant random access to multiple bits via a wavelength-division multiplexing scheme. Based on a three-dimensional finite-difference time-domain method, we numerically illustrate a compact device capable of switching and accessing 8 bits in different cavities with a 5 nm wavelength spacing in the conventional (C) telecommunication band. Our multi-bit device holds promise as a new paradigm for developing an ultrafast photonically-addressable spintronic memory and may also empower novel opportunities for photonically-driven spintronic-based neuromorphic computing.