See About using video for recommendations on video formats. Basically, you should only use either MP4 video files or online video sharing services (YouTube, Vimeo and Prezi). You can also use Flash SWF video in CHM files. All other video formats will cause problems sooner or later.
Check that the correct codec and/or player are installed. Videos can only be played if the correct player and codec support for them is installed on the computer where your help is being viewed. So it's a good idea to only use common formats like Flash and Windows Media that most users will have support installed for.
With some formats and/or browsers you may need to activate the Start Automatically and Show Controls options in the Insert Video dialog for the animation or video to play correctly. Flash videos do not support these external controls – if you want player controls for Flash you must embed them in the video with the program that you use to create it.
On some systems you may find that need to be working in a Windows account with full administrator rights to use the movie preview function. If the function doesn't work and you know that the correct player is installed try switching to a full administrator account.