No you shouldn't be concerned and you might have changed the filter prematurely. Look at the filter with a light behind it. If you can see through it the engine can breathe through it. Changing a filter just because it has that little bit of black is not cost effective, neither is changing a filter just because it has XXXXX number of miles on it. If it isn't dirty, it isn't dirty. Obviously if you can't see through half of the filter, by all means change it.
Why would that concern you? The only way that is likely to occur would be if the air box or the induction tube were disconnected. That would be readily apparent upon inspection but would likely be known much earlier by an odd whooshing sound from under hood or other noise.
It could just mean that the vehicle is being operated in non dusty and dirty conditions.