Advancements in neural rendering techniques have sparked renewed interest in neural materials, which are capable of representing bidirectional texture functions (BTFs) cheaply and with high quality. However, content creation in the neural material format is not straightforward. To address this limitation, we present the first image-conditioned diffusion model for neural materials, and show an extension to text conditioning. To achieve this, we make two main contributions: First, we introduce a universal MLP variant of the NeuMIP architecture, defining a universal basis for neural materials as 16-channel feature textures. Second, we train a conditional diffusion model for generating neural materials in this basis from flash images, natural images and text prompts. To achieve this, we also construct a new dataset of 150k neural materials. We demonstrate real-time decoding performance for our universal materials at 1024 x 1024.