Idea for an easier hook for translations

[John Doe123]

I was thinking of a python script that would read japanese text off the screen through a screencap, run it through an OCR to get the raw text, and then translate it and display it on the screen.

I'm not a professional programmer by any means, but I think the first two parts (reading text from the screen and the OCR) should be possible. PIL should do the first and pytesseract should be good for the second part. I haven't looked into the third part, but that should be trivial compared to the first two.

Thoughts?

 

[Beatru]

Hi,
I've learned 9 different programming languages and wrote hundreds of thousands of lines of code in my life. I've participated in, and won several hackathons. So I think I can give some input.

Getting the computer to read text from an image (regardless of font being used) is possible but I'd expect that to be something someone could spend an entire college thesis on. Specifically if you are doing it from scratch.

Having said that, I wouldn't be surprised if someone already spent the time to create it. If you google search "python read text from image library" you will probably find something.
Yup found this:

You must be registered to see the links

That article even explains how to set it up in like 5 minutes.

You must be registered to see the links

is the name of the library that you would use.

Edit:
For replacing images, maybe try Python Imaging Library (PIL)

 

[Beatru]

On second thought, these are English libraries and may not work with Japanese text. You'll have to try it out and see what happens

 

[habisain]

This sounds a lot like a less featureful version of Capture2Text:

You must be registered to see the links

I'm not saying that there isn't a market for tools like this, but I think you'll have to come up with a better distinguishing point. Also the reason why that Hook-ing into an executable is preferred is you get the actual text easily. OCR is computationally expensive and error prone.