The relationship between entropy ($S$) and degeneracy ($W$ or $g$ depending on the book) $$S = k\ln(W)$$ is one of the fundamental equations of statistical mechanics. But where does it come from? Or more precisely, why $\ln(W)$ and not, say, $\sin(W)$? And why does $k$ have units of J/K?
I believe it goes back the second law of thermodynamics, i.e. it is based on observation. The second law can be stated in many ways and one is that heat ($q$) spontaneously flows only from hot ($T_h$) to cold ($T_c$) bodies.
Mathematically this can be stated as $$\frac{q}{T_c}-\frac{q}{T_h}>0$$ or $$\Delta S >0$$ where $S$ is defined* as $$dS=\frac{dq_{rev}}{T}$$ This establishes the units of $S$ and, hence, $k$.
Furthermore, since $q$ (like all energy) is additive, $S$ must be additive: $S_{total}=S_A+S_B$. However, $W_{total}=W_AW_B$, so $S=kW$ won't work. However, $S = k\ln(W)$ will.
The final question is now whether the logarithm is the only mathematical function for which $f(xy)=f(x)+f(y)$. It turns out that it is** if we require the function to be continuous (thanks to Niels Grønbæk for help here), which we do since $S$ as defined by the second law is non-discrete like the energy.
Another important property of $\ln(W)$ is that is has a maximum value when $W$ is largest. So the most probable state will have the largest entropy.
* This definition begs the question: if heat is transferred, what $T$ do you use, the one before or after the heat transfer? The answer is that $dq$ has to be so small that $T$ is not affected. Since $T$ is not affected this is a $rev$ersible process. Furthermore, notice that this law also introduces the concept of temperature.
** Se also http://www.physicsforums.com/showthread.php?t=566358. Not that I understand all of it!
Related blog posts
Illustrating entropy
Entropy, volume, and temperature
I believe it goes back the second law of thermodynamics, i.e. it is based on observation. The second law can be stated in many ways and one is that heat ($q$) spontaneously flows only from hot ($T_h$) to cold ($T_c$) bodies.
Mathematically this can be stated as $$\frac{q}{T_c}-\frac{q}{T_h}>0$$ or $$\Delta S >0$$ where $S$ is defined* as $$dS=\frac{dq_{rev}}{T}$$ This establishes the units of $S$ and, hence, $k$.
Furthermore, since $q$ (like all energy) is additive, $S$ must be additive: $S_{total}=S_A+S_B$. However, $W_{total}=W_AW_B$, so $S=kW$ won't work. However, $S = k\ln(W)$ will.
The final question is now whether the logarithm is the only mathematical function for which $f(xy)=f(x)+f(y)$. It turns out that it is** if we require the function to be continuous (thanks to Niels Grønbæk for help here), which we do since $S$ as defined by the second law is non-discrete like the energy.
Another important property of $\ln(W)$ is that is has a maximum value when $W$ is largest. So the most probable state will have the largest entropy.
* This definition begs the question: if heat is transferred, what $T$ do you use, the one before or after the heat transfer? The answer is that $dq$ has to be so small that $T$ is not affected. Since $T$ is not affected this is a $rev$ersible process. Furthermore, notice that this law also introduces the concept of temperature.
** Se also http://www.physicsforums.com/showthread.php?t=566358. Not that I understand all of it!
Related blog posts
Illustrating entropy
Entropy, volume, and temperature
