This is the easiest way to remove labels from glass bottles! 1. Hot water 2. Large container 3. Baking Soda or White Vinegar 4. Glass bottle 5. Sponge Step 1 Take a large container and fill it up with hot water. Sprinkle some baking soda in the water and leave it for five minutes. Step 2 Now, dip the glass bottle in the solution in a way that the label is dipped properly in the water.