Similarly, as I understand it, "e" is pronounced differently in "e ge ke le ne" on the one hand and in "ye" on the other (sometimes spelled "ê" in the second case) - it's more of an "uh" in the first set and an "eh" in the second.

And "u" is pronounced differently in "yu ju qu xu" (like French "u") compared to "wu zhu chu shu" (like French "ou") - this is mostly an orthographic convention: since the /u/ sound can't appear after j- q- x-, they can get away with dropping the dots on "ü", whereas since both /u/ and /y/ can occur after /l n/, there you have to spell "nü, lü" explicitly to contrast them with "nu, lu".