This is original! An NPC that comes in a bottle. I like this idea..
I wouldn't bother with the phylacteries. Its magic - no need to create a storage container for memories, and I think it unnecessarily complicates the core idea. If you must, perhaps it takes over a small portion of the pill-popper's own mind to store its memories, which could then potentially 'leak' into the user's own consciousness. Go to Comment
Okay, armour created from Ivory is not too far-fetched - lacquered wood and boiled leather both make reasonable armours. I could see alchemists involved in the preparation of the substance but not in the actual construction of the armour.
Apart from Moons comment on Ivory being bone (its not, similar, but different) his comments are spot on.