It all starts with an explosion that creates a Pyrocumulus Cloud. This ball of burning hot gases is accelerated outwardly in all directions. Since the burning ball of accelerated gases is hotter, and therefore less dense, than the surrounding air, it will begin to rise- in the case of nuclear explosions, extremely rapidly. This ultimately forms the mushroom cap.
As the ball rises, it will leave behind air that is heated, creating a chimney-like effect that draws in any smoke and gases on the outer edge of the chimney- convection in action! Visually, this forms the stipe (stalk) of the mushroom.
The perception that the mushroom cap is curling down and around the stipe is primarily a result of the differences in temperature at the center of the cap and its outside. The center is hotter and therefore will rise faster, leaving the slower outer edges to be caught up in the stipe convection’s awesome attributes.
Once that cloud reaches a certain point in our atmosphere, where the density of the gas cloud is the same as the density of the surrounding air, it will spread out, creating a nice cap.
This brings me to the shorter, yet more geeky answer.
This entire process is something that describes the Rayleigh-Taylor instability. This instability is well known in physics and, in general, describes the merging between two different substances (mainly liquids and gases) that have different densities and are subjected to acceleration. In the case of an atomic bomb, the acceleration, and the hotter gases creating the differing densities of material, are caused by the explosion.
From this, you might have guessed you don’t necessarily need an atomic bomb to create a mushroom cloud. All you need is enough energy delivered rapidly (in this case an explosion) that creates a pocket of differing densities of material (in this case, heated gases).
There are numerous other examples in our world that create, and are described by, the same phenomenon that gives us this formation. For instance, the magnetic fields of planets, the jet-stream of winds that help control our planet’s climate, the sound of snapping shrimp, even our understanding of certain different forms of fusion can all be attributed to Rayleigh-Taylor instability.
Now, you might have also noticed that nuclear explosions, besides producing this frightening fungal formation, also sometimes result in a cloud ring around the mushroom cap. What’s going on here is that a low pressure area is created via the negative phase of the shockwave (the phase that follows the wave of compressed gases at the leading part of the shock wave). This results in a drop in temperature, which along with the low pressure can potentially lower the dew point sufficiently for a temporary cloud to form. This cloud halo around the explosion is known as a “Wilson Cloud”, named after Scottish physicist Charles Wilson who invented the Wilson Cloud Chamber where similar sorts of things can be observed.