I've been on Atkins for about 3 weeks.. and I've lost about 20 pounds..?

..and I just ate a whole bunch of bread! Right when I was starting to incorporate more carbs slowly.. Is this going to mess up the whole process and make me have to start over? Or should I just keep going with the small regular increments?