As a girl dad of two (currently 5 and 11), I’ve been through this exact “4T vs. 4” struggle more times than I care to admit, and I’ve learned the hard way that there are some small but important differences between the two.

The biggest thing I noticed with my kids is that 4T is designed for toddlers who still wear diapers. It’s shorter, wider, and often (not always) cut with extra room in the seat for a diaper or pull-up. So as long as my girls were in diapers, we went with the 4T size whenever it was available.
Once my oldest hit that preschooler growth spurt and finally ditched the diapers, we made the jump to size 4. In my experience, size 4 is really not dramatically different but, in most brands, the legs will be a smidge longer and the butt area won’t be as baggy. These clothes are cut slimmer through the waist because they assume the kid is wearing actual underwear, and they usually add an extra inch or so of length.

My rule of thumb for my kids has always been: if they’re still in diapers, stick with 4T. If they’ve moved into “big girl” undies and are starting to look like a mini-adult, grab the 4.
Just a heads-up that every brand fits differently, so I always hold the hangers up against each other in the aisle just to be sure or check the official brand size chart. Usually, you’ll see the 4 is just the “tall and skinny” version of the 4T, but only by an inch or two or maybe even less.
For more, read: 5 vs 5T
Hope this helps!