After years of not playing and I lost my old save, so I decided to replay again and lelearned about Devil King route patch at the end of the game.
I applied the patch that enables to gather Devil items. Didn't think it would work!
So yeah, to collect Devil items/enter Demon King route, speak to the "Father" in church, choose Shut Down (+10% corruption) then follow him to the dungeon and defeat him there. After the battle, try to examine the statue (nothing happens without the patch) then you can collect the first two Devil items from the cave you fought Makina and the cave you found the herb.
After collecting the first two Devil items you can get the third one by defeating the final boss. Just after the final showdown with the mad prince, you have option whether to kill him or spare him. You'll get the third item from him either way, but if you spare him then he'll join you as additional party member!
The next destination is the deepest part of Devil Ruins to confront the true final boss. With all 3 Devil items, you can finally open the sealed door that leads to the true final dungeon. The mobs there are quite tough compared to all other dungeons so far (though I haven't visited the dungeon that blocked by a boulder which requires not recruiting Nina).
After defeating the true final boss, once again you have option whether to kill or spare him (the mad prince). I end up sparing him.
Still, I don't figure out how to recruit Klaus. I wonder if Eris has to first meet him at the castle gate (which I didn't). Though he also appeared in the ending of my current playthrough.
Anyway after completing my first playthrough, I found Gallery was fully unlocked! I swear before then, the only unlocked pictures are at the beginning of the game and when losing to the mad prince in the final battle. During my first playthrough I may viewed some H scenes by (purposely) losing in certain battles however they are all not "registered" in the Gallery as I didn't save the game after viewing H scenes. Only H scenes from losing the final battle is the exception as it brings you directly to the title screen.