It's a shard of the mirror that was given to Harry as a birthday gift from Sirius. Harry only unwrapped the gift, and therefore discovered the mirror after Sirius' death, along with the note that was on the back. The note stated that, with the mirror, Harry would just need to call out Sirius' name and Sirius would hear him through the matching mirror. Sirius could then pick up the matching mirror and they could speak, no matter how far apart they were. Harry, emotional and probably a little bit disgusted with himself for only discovering the mirror after the events that led to Sirius' death, throws the mirror in his trunk where it shatters. He carries it with him later, probably to keep a piece of Sirius with him, but also because he believes that he sees Dumbledore's eye in it after Dumbledore's own demise.