The Stupa of Bodhnath, Nepal. Photo from Andrea's photos on Flickr.
From the Shambhala Mountain Center website an explanation of the symbolism of the stupa.

This is a Tibetan stupa. The one from Bodhnath is a Nepalese one closer to the Indian style.
The basic structure of a Chorten (Tibetan name for stupa) consist of a square foundation symbolizing the earth, a dome symbolizing water, and thirteen tapering steps of enlightenment symbolizing the element of fire. These steps lead to a stylized parasol, the symbol of wind, which is topped in the ethereal sphere by the well-known ‘twin-symbol’ uniting sun and moon, which is the shimmering crown of the Chorten.
The Analogy with the Symbolism of the Stupa

The Seed of Highest Enlightenment, also depicted as a Tongue of Flame (Bindu) to be realized above the double symbol crowning Chorten.

The double symbol (Surya Chandra) of Sun and Rising Moon is an emblem of the Twin-unity of the Absolute Truth (of the sphere beyond normal comprehension) and the Relative Truth (of the worldly sphere).

The stylized Parasol (Chattra) symbolically giving protection from all evil.

The thirteen Steps of Enlightenment, i.e. the first ten Steps of Enlightenment (Dasha-Bhumi) and the three higher levels of supra consciousness (Avenika-smrityupashthana).

The dome, corresponding to the primeval mound, as Receptacle of Relics or offerings (Dhatu-Garbha); the dome-line edifices of Old Indian Stupas were also called egg or water-bubble (Budbuda).

The base (Parishada) is square and four-stepped, its sides facing the four directions. Analogous to the underworld.

The symbolic relationship between square and circle is that of the human and the divine. The integration of the two is a metaphor for equilibrium between earth and heaven.

Orientation towards the rising sun is a vital factor in assuring the effectiveness of a sacred place. When constructed, the temple must not only be placed at the Center of the World, it must be aligned with the four cardinal points of the compass.
To this day, all Mosques and Cathedrals are oriented, which is to say that they face the East or the rising sun.