I was looking through my visitor statistics earlier today and noticed that nine people have found my blog through looking for the exact phrase "When should I start walking my Border Collie", in the last two days. Given that it's such a popular question, I'll do my best to answer it, according to my own experience.
I believe that a Border Collie, like any other dog, should begin walks as soon as it has had all of its puppy inoculations. (In England, this is usually around twelve weeks of age.) I can see no real merit in delaying things further.
It is important to remember that dogs must not be walked out in public, before their inoculations, in order to avoid the risks of parvovirus and distemper.
However, before your pup has the inoculations, there is nothing stopping you from carrying your puppy outside, in your arms. I strongly advise that you do this, to get it used to sights, sounds, and smells, prior to being able to walk outside.
Before your dog is able to be walked outdoors, you can spend the time getting it used to wearing a leash indoors and around your own garden (presuming your garden isn't used by other dogs). Some puppies hate the leash for the first time, some take to it like a duck to water. Make sure the leash wearing is always supervised, and that it is a lightweight leash to begin with.