Right, but you are not opposed to the idea, just who sits in the place of moral authority. If I read what you are saying correctly, you are stating that God has a caste system. If this is true, then in your belief system, there is nothing wrong with a caste system if applied benevolently.
Even though you state you are against racial caste, I would ask you why? Why would race make a difference if you are fine a caste in every other way? That seems like selective reasoning to me. I am not sure being for slavery has any grey area? I am not trying to put words in your mouth, but better understand what you are saying.