Child Visitation question in Texas.?

I have two children with my soon to be ex husband. One is 5 and the other is 20 months. We have an order with the attorney general so all the child support and visitation is set up with them. Since the order has been in place he has only taken our 5 year old. Never once has even taken our youngeat not even for a day. Well now hes threatening me that hes going to start keeping our youngest just because I got mad at him cuz he was very late in bringing our oldest back on his last weekend. I am very lenient with him. Hes suppose to drive an hour to pick them up and drop them off but im nice enough to bring her an hour away to him. Any time hes needed her and it wasnt his time ive always let him have her. But never once has he taken the baby. My question is. Even tho its not stated in our papers that he gets over night vists with our youngezt can he? Someone told me that they have to be 3 before they can get over night vists. Im so nervous because my youngest is literally attached to my hip and is never away from me and has alot of stranger anxiety. So if anyone can tell me what my options are and if the 3 year old over night vists arent the way it is, what i can possibly do to change it quickly??? We luve in texas. Thanks everyone.