Using reference images

I'm trying to create a generator that will take an uploaded reference image and create new images using that same character.

I can't for the life of me get it to actually use the reference image in it's generation.

Is there some trick to it or something?