It shouldn't matter for any still layers, it just looks confusing. It's usually only relevant when using motion stuff or features like time-remapping. not so much that it technically is problematic, but you simply lose track of what the actual "time" is for a given element, e.g. in a pre-comp.
Mylenium