I fractured my ankle and the doc put it in a cast for 3 weeks and said I can start walking on it whenever?

I fractured it yesterday so Im thinking about waiting a week until I start trying to walk on it and add weight. Its a small fracture maybe 1/4 through the ankle. Would waiting a week be a good idea? Or should I wait a little longer? They really didnt tell me when I should start walking on it. Is it worse to wait or should I start adding weight asap?