don't think the xbox supports hd videos unless they are in avi or wmv hd.
i'm guessing your hd movies are mkv, in which case you need to encode them to work.
On the ps3 you can't play them either but you can convert them into a vob and play them that way, no encoding required. Don't believe the 360 works in the same way though, well i haven't managed to get it to work like that anyway.