It's pretty easy, really.
First, unzip the character. You may have to remove the folder that has the actual files from the folder that appears after unzipping.
Next, go to the Data folder and open up select.def in Notepad or a similar program. If you scroll down a little, you should see a comment section that says at the end of it something like "Insert your characters below". Below that line should already be the characters your version defaults with (mine started only with Kung-Fu Man). Below that comment line, copy the name of the .def file of the character you want to enter. For example, let's say I have Dante here and I want to add him to my character list. So I would open up select.def, which is in the Data folder. Then, having unzipped Dante into my Chars folder, I would open up the folder and find the .def file inside it. In this case, it's dante.def. I would then go to a new line in select.def and type in "dante" (no quotation marks). Note that I don't put .def at the end of this. I then save the file and can now use Jotaro in Mugen.
You can usually assume that the .def file's name is the same as the folder that holds it, but this isn't always true, so it's a good idea to check just in case anyways. And remember, once you unzip the file, you may have to open the folder that appears. Often, for me at least, the files are actually inside another folder inside the first, and I have to move that folder to the Chars folder before the character will appear there.