Not sure if you can for goblin, but a guildie had a problem when he did it with a worgen. The game wouldn't let him accept a summons while he was still in Gilneas.
I know nothing about goblins... but the worgen area was phased initially... so the 2 seater mount might not work in all areas of the starter zone.
You cannot be taken out of the starter zone and no one from outside can access it as it is phased. This includes mounts/summons. The only way is to make a toon, level beyond 10 (maybe) and then have it race-changed to skip the starter zone.PS: I have leveled 2 goblins so far.
Make sense for gob/worgen. Every other class you can, but you wouldn't be able to start any other questline unless you moved out to barrens/westfall.