First things first:
thank you for developing these nice tools, imho they are very useful to anybody serious about language learning nowadays.
My use case:
I want to read Manga and play games in Mandarin Chinese (Simplified), but I am testing it mostly for the second use case.
I have been testing it mostly with the game "Chinese Parents".
Disclaimer:
I have developed a chrome extension and a win app for myself, but they are very basic, ugly and strictly for personal use. I have no intention at all to develop them further and to make them public at any future time, ever. So, if I refer to them is only as an example to show features I found useful.
I hope LookUpper comes to the point where I can avoid using my own apps altogether.
Part 1 - Feedback on the actual state of the app
A) The good stuff
I love how fast LookUpper works. Even in the accuracy mode, it's very fast. Incomparable with sending a API call to an AI.
This is a big thumb up.
It also gets a lot of stuff right. Not everything, but a lot.
I also like the customization options, kudos on that!
B) OCR and Translation accuracy
Sometimes the OCR doesn't recognize the right characters and/or the translation doesn't work.
For example, the very first example sentence has a clear mistake:
Of course it's not companies, and the dictionary gets it right, it's 词 (word) and not 司 (part of the word for "enterprise"). But the translation gets it wrong.
Sometimes the OCR goes completely crazy, like:
Or the translation doesn't make any sense (should be to English)
Or the translations leaves out parts, important ones. Compare these two version:
You'll notice that in the first one, the word "expectations" is completely missing, but that's a very important part of that short sentence.
C) Vertical text
A situation where I think it's obvious what's going on and maybe can be fixed, is when the text is vertical.
This is pretty common in Chinese (and Japanese).
I couldn't test it extensively with mangas, where the text is mostly vertical, because I finished my free tokens 😃 (and I am not sure it's worth for me to switch to Pro at the actual state of development).
But I think it's mostly struggling with vertical text.
Examples from manga and game:
In my applications, I send the screenshot of the part I am interested in to ChatGPT Vision API, mentioning that the text could be vertical and right-to-left. This gives me good accuracy.
But this method is slow and requires the use of tokens, so it's not viable for LookUpper.
I have almost no experience with OCR, but here they are my 2 cents for brainstorming.
If the OCR allows you to flag vertical vs horizontal text, then there are two options:
1) leave the user the option to switch (so, human recognition of orientation)
2) let the OCR run in both directions and check what makes sense by comparing it (not sure if feasible algorithmically without AI)
Part 2 - Suggestions
D) Transliteration
The single most important addition would be transliteration in pinyin (and possibly also the option to use other transliteration systems). Because in Chinese and Japanese, if you don't know how to pronounce a character, you just don't know it. You need to look it up. And this is word-based (same character can have multiple pronunciations).
In my extensions, I do it like this:
or this:
So I do it for the single word AND for the whole sentence.
I find them both very useful, but the transliteration of the word is absolutely necessary.
To use the dictionary in LookUpper isn't a good substitution, for two reasons: 1) it shows only one character at the time, 2) as I already pointed out, the pronunciation is often contextual.
E) Word translation + Definition in two different languages
Right now, in the "definition" section, I see:
the Chinese word
a definition (in a language of choice)
I think it'd be better to have:
the Chinese word + the transliteration + the translation of the word to a language of choice
a definition (in a language of choice)
a definition (in another language of choice)
In this case I'd choose to have the definition in Chinese and in English
because sometimes I may not understand part of the definition in Chinese, but on the other hand, to read the definition in Chinese is good training
The End.
I think I gave all the feedback I had for today 😃 (I hope it wasn't too much)
Thank you again for developing this wonderful tool!
I look forward to your answer and to contributions from other users.
For any further questions, testing and feedback, I am at your disposal.
