I’ve messed with this dozens of times with various AI models that are generally good at abstractions with advanced prompting, custom diffusion settings outside of the typical, and some hacks in the model loader code. I seem to lack the vocabulary to describe the fundamental geometry of centrifugal gravity in the kind of depth required. Like how does one say that the road ahead curves up like a hill continuing overhead with the buildings anchored to…
I need language here that has unambiguous specificity and likely does not occur in any other context. Layperson verbosity will fail to get traction without fine tuning the model first. I prefer to explore what models can really do using settings and the prompts only.
Most of my vocabulary and understanding of geometry is limited to the Cartesian planes and CAD assemblies. Perhaps someone here has a better lexicon and doesn’t mind sharing.
(Post image is from a Blender rendered video someone posted on YouTube about what life in an O’Neill cylinder might look like)
Diffusion models have a very limited understanding of language compared to modern LLMs like GPT4 or Claus, etc.
https://huggingface.co/docs/transformers/model_doc/t5
Most likely use something like Google’s t5 here. This is basically only meant to translate sentences into something a diffusion model understands. Even chatgpt is just going to formulate a prompt for a diffusion model in the same way and isn’t going to inherently give it any more contextual understanding.
The simple answer is they are simply not there yet for understanding complex concepts. And I suspect that the most impressive images of impossible concepts they can drum up are mostly by chance or by numbers.
deleted by creator
Nevertheless, these models are trained with broad yet shallow data. As such, they are glorified tech demos meant to wet the appetite of businesses to generate high value customers who could further tune a model for a specific purpose. If you haven’t already, I suggest you do the same. Curate a very specific dataset and very clear examples. The models can already demonstrate the warping of different types of lenses. I think it would be very doable to train one to better reflect the curving geometry you’re looking for.