Ttgjyuk

How do the sound waves compare between different octaves? Why is it that you can play a low c and a c, and they are at different frequencies, but the same tone? An octave above a note is the fourth overtone, I'm assuming, so would the slopes of the waves align? You can still tune to these notes between octaves, and the beats can still be heard if out of tune. My AP physics teacher was unable to answer why.

The phenomenon you're asking about is known as octave equivalence. It's a hard-wired thing in the human ear-brain system. (We know it's hard-wired because it's present without musical training and is true cross-culturally.) Notes that differ in frequency by a factor of 2 (or a power of 2) are perceptually similar, and may be mistaken for one another, even by trained musicians. So this is really a fact about the ear-brain system (i.e., psychology and physiology) rather than physics, although it does relate to a physical property.

Looking through the index entries on this topic in Diana Deutsch, The psychology of music, 3rd ed., 2013, I don't see anything about any physical, neurological, or evolutionary explanation of octave equivalence. That may mean that nobody has such an explanation, or just that it wasn't something that Deutsch wanted to get into in this anthology of articles. It seems likely that it has at least some physical basis, because periodic tones usually have their first two frequencies (fundamental and first harmonic) in the ratio of two to one.

An octave above a note is the fourth overtone, I'm assuming

No, it's the first overtone.

How do the sound waves compare between different octaves?

Pitch perception can be complicated in some cases, but essentially our sense of pitch is normally based on the period of the wave. So notes differing by an octave in pitch have waveforms that are related by the fact that their periods are in a 2 to 1 ratio.

You can still tune to these notes between octaves, and the beats can still be heard if out of tune.

The beats would be beats between the first harmonic of the lower note and the fundamental of the higher note.

I basically agree with the answers of Ben Crowell and of Pieter. The fact that we hear the octave as a pleasant pure interval is hard-wired in the human hearing system. As Pieter explained, musical instruments produce tones with many overtones of different intensities, where usually the frequency assigned to the tone is the lowest, and the overtones are integer multiples of it.

The point I can add to these previous answers gives a hint, why the octave is a very special interval sounding most pure. The point is that if you produce a perfect octave on a musical instrument, the frequencies of all the overtones of the higher note will exactly line up with frequencies of the overtones of the lower note. This means that any beating in the resulting sound is reduced to a minimum.

The situation is already different for a perfect fifth. In this case only every second overtone of the higher note lines up with an overtone of the lower note.

The situation gets even worse for the other intervals. For example, for the minor third, only every fifth overtone lines up, and other overtones start to generate beatings. For dissonances, like the minor second, none of the overtones line up, and severe beatings arise from the overtones, resulting in a 'rough' perceived sound.

The human voice and most musical instruments produce tones with many overtones. The sound pressure is periodic with a period $T$, but does not vary like a sine wave at all. Instead, it can be described by a Fourier series. That is the sum of sines that are integer multiples of the fundamental frequency $f=1/T$.

So when a singer sings with a fundamental at 220 hertz (an a), that note includes overtones at 440 hertz (a'), 660 hertz (e"), 880 hertz (a") etcetera. This is why beats can be heard between two instruments that play an a and an a' slightly out of tune.

would the slopes of the waves align?

The ear does a kind of Fourier analysis of the input waveform, where different frequencies are mapped to different parts of the cochlea and to different nerves going to the brain. Our hearing is almost insensitive to the relative phase between the frequency components.

One can look at the frequency content of sounds with spectrogram apps on mobile phones or on https://musiclab.chromeexperiments.com/