[GSoC] Implement the CSV importer

[BrayanDSO]

Purpose / Description

On top of #12072

Fixes

Closes #6772
Closes #9437

Approach

This implements the base for using upstream's HTML/Js pages directly on AnkiDroid.

I've used krmanik#23 as base of this PR. I wouldn't have managed to do it without all the previous work of Mani and Damien, so I thank them very much.

As discussed on krmanik#23, to use an Anki HTML page, we should be able to intercept POST requests. After some proof of concepts made by Mani, the conclusion was that hosting a local server, like the desktop application, may be most suitable approach.

To create the local server, I've used the same approach of krmanik#23 and added NanoHTTPD, a java server library, as dependency. I don't have experience with any Java server libraries, so I can't tell with confidence if this is the best library, but it seemed solid to me and already had a working model on Mani's PR, so using it looked like the best choice for the moment.

Besides the server, this PR implements the base classes for creating AnkiPages. Each page can easily extend PageFragment and define what it needs. I've tried using the new Statistics, Card Info, Deck Options pages and they work well. Try https://github.com/BrayanDSO/Anki-Android/tree/csvImporterOtherPages if you want to test them (the deck options screen may need some extra work).

Finally, the CSV importer page and its endpoints are implemented and can be used with the new backend.

PENDING (for future PRs):

• Close the Import dialog after the taps "OK" on the progress dialog
• Inject CSS to match the themes' colors
• Maybe hide the file path with CSS/Js, because it may be big and there isn't much horizontal space in phone screens on portrait mode (already on upstream)
• Not for my GSoC project, but the other screens could be implemented too, as they are mostly done, so other people could test/work on them as well

How Has This Been Tested?

NOTICE: this needs an updated WebView (77+) to work correctly (see #6772 (comment) and the comments below for discussion), which shouldn't be a problem as the project's minSdk (21) supports it. But if you are testing on a emulator, be aware that the WebView shipped on some older SDK versions may be an old version, so either update the WebView or use a newer SDK.

NOTICE 2: there may be a flickering issue after remapping fields, but that was already solved on upstream (ankitects/anki#2005)

1. Enable the new backend (Settings > Advanced)
2. Deck picker > Three dot menu > Import > Text file
3. Select a CSV file (I've tested with Anking, pretty heavy and popular deck)
   • Here's a mini version (23KB) if anyone wants to test it: MiniAnking.txt
   • And here's a slightly cut version (GitHub only allows files up to 25MB):
     Anking.txt
4. Configure anything on the CSV importer
5. Tap Import

Screen_Recording_20220821-073427_AnkiDroid.mp4

Checklist

Please, go through these checks before submitting the PR.

• You have not changed whitespace unnecessarily (it makes diffs hard to read)
• You have a descriptive commit message with a short title (first line, max 50 chars).
• Your code follows the style of the project (e.g. never omit braces in if statements)
• You have commented your code, particularly in hard-to-understand areas
• You have performed a self-review of your own code
• UI changes: include screenshots of all affected screens (in particular showing any new or changed strings)
• UI Changes: You have tested your change using the Google Accessibility Scanner

[dae]

Nice!

[krmanik]

Should we move last two commits to separate PR for better diff in this PR?

[BrayanDSO]

Should we move last two commits to separate PR for better diff in this PR?

Good idea. Done it.

[mikehardy]

Maintainer note (or, other maintainer note, since dae and krmanik are both here already): I was really curious about both the localhost server and the HTML/JS from the backend work and have also participated in PR 23 on krmanik's fork while this was developed.

TL;DR: localhost server really seems like the best approach after testing, and I'm super-excited to get HTML/JS support from the backend. This PR is pretty amazing in the forward progress it represents for AnkiDroid

[BrayanDSO]

Updated the PR's description with a new video, now showing cancelling and undoing the import as well.

[david-allison]

The future is now! Incredible work

[BrayanDSO]

Only squashed/simplified the last commits with the last push force

[dae]

Nice work on adding undo!

[dae]

Maybe hide the file path with CSS/Js, because it may be big and there isn't much horizontal space in phone screens on portrait mode

Already coming: ankitects/anki@2e2fc79

[mikehardy]

I don't have anything actually blocking this. I'm most happy about the monumental collaboration here on all levels

David spearheading then everyone else working to get Kotlin going
Damien and Mani's efforts to get the new version of the backend up and plumbing Anki JS through
Brayan, all the work here

[krmanik]

I have pulled the PR locally and tested it on an actual device. I have tested import feature using the following files.

1. Anking.txt, provided by the Brayan
   File Size: 24 MB
   Lines in file: 20k
   Note Type: Basic
2. Tatoeba Chinese English sentences pair
   File Size: 5 MB
   Lines in file: 63k
   Note Type: Basic
   I have interrupted import after 20k notes to test interrupt.

I am happy with the result of the import-csv feature. But the only issue I am seeing is that only the text file is selectable to import, other file (csv and tsv) is not selectable. I renamed files to test import.
So, may be follow up PR for making the csv and tsv selectable should be fine.

This is one of the best features coming to AnkiDroid. Thanks to @BrayanDSO.

[mikehardy]

Let's merge this to unblock all the stuff piled up on it (multiple follow-ons, that is), and the csv file selection issue can be a follow-on as well

[BrayanDSO]

I am happy with the result of the import-csv feature. But the only issue I am seeing is that only the text file is selectable to import, other file (csv and tsv) is not selectable.

lol. Learned something today. I'd thought that Android imports were based on MIME types, so if it works with .txt, renaming it to .csv shouldn't change anything, which means I'm probably missing something.

But that is easy to solve anyway