root/pleroma-ebooks
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
206 commits 1 branch 0 tags 262 KiB
Commit graph

206 commits

This branch
This branch
All branches
Author SHA1 Message Date
io
dce92186f9 config.defaults.json: add db_path key 2021-07-26 06:17:28 +00:00
io
a3348602c1 fetch_posts: add KeyboardInterrupt handling 2021-07-26 06:17:01 +00:00
io
4ba3bb1662 fetch_posts.py: use argparse 2021-07-26 06:04:32 +00:00
io
b8e727cfd4 fetch_posts.py: remove unused import 2021-07-26 06:04:14 +00:00
io
5dcbd4ccfa add generation_mode to config.defaults.json 2021-07-26 06:03:43 +00:00
io
8be760e8af broaden .gitignore 2021-07-26 06:03:29 +00:00
io
6b97f6cbff split requirements.txt into gpt2, markov, and base reqs 2021-07-26 05:45:45 +00:00
io
960061b92b remove old app.json 2021-07-26 05:35:14 +00:00
io
a78c60ac6f remove posts.db from git 2021-07-26 05:19:31 +00:00
io
330fdc2809 rewrite fetch_posts.py from scratch 
now it should be properly async by fetching posts in a separate task and sending them across
a queue to a task that inserts to the DB
 2021-07-26 04:59:04 +00:00
io
c22a493dff license compliance stuff and other changes 
relicense to AGPLv3
comply with the MPL's file copyleft
some other changes in the direction of supporting GPT-2 which were hard to untangle from the rest of the changes (sorry)
 2021-07-26 04:52:44 +00:00
io
944e2fc3a5 use json5 instead lol 
toml sucks add null when
 2021-06-18 10:38:58 +00:00
io
79301382cd fix AttributeError in Pleroma.reply() 2021-06-18 09:05:55 +00:00
io
b5b4349d7f implement chud alarm 2021-06-16 04:58:24 +00:00
io
213bdf75c3 add asqlite to requirements 2021-06-16 04:34:59 +00:00
io
fcbbd7decf fix requirements.txt 2021-06-16 04:30:56 +00:00
io
8c14e031e8 remove Dockerfile 2021-06-16 04:26:36 +00:00
io
a6aff9b4fa update README 2021-06-16 04:26:29 +00:00
io
5f9959fff8 rename main.py to fetch_posts.py 2021-06-16 04:26:22 +00:00
io
b93ee8fa45 fix readme link 2021-06-16 04:23:47 +00:00
io
8c0041e8ad update README 2021-06-16 04:17:57 +00:00
io
36a416da7a fix Pleroma.reply() not tagging 2021-06-16 04:14:18 +00:00
io
e273a28aae update README 2021-06-16 03:50:27 +00:00
io
01a39db9d6 rewrite reply.py too 2021-06-16 03:49:34 +00:00
io
5d1c3397b6 rewrite for anyio+aiohttp 2021-06-16 01:59:57 +00:00
io
93095f62f3 relicense 2021-06-16 01:29:53 +00:00
io
16a39c8493 update README 2021-06-14 21:43:16 +00:00
io
04178b37d7 fix scraping posts 
saves the cursors provided in the first page to the db so that we can reuse it next time we fetch,
instead of assuming the format of the cursor URL manually using min_id
 2021-06-14 21:39:17 +00:00
io
fe1474ffd0 call raise_for_status() on all GET requests 2021-06-14 20:34:33 +00:00
io
a46d7fe95c SQL NULL a fuck 2021-06-11 21:37:09 +00:00
io
71dbf59796 add ability to ignore CWs 2021-06-11 21:29:51 +00:00
Agatha Rose
a904587b32
Clean up formatting and help linter calm down 2021-06-05 00:38:36 +03:00
Agatha Rose
dd78364f2d
Expose overlap ratio and length limit to config 2021-06-05 00:14:56 +03:00
Agatha Rose
54563726b2
Add testing virtual env to .gitignore 2021-06-04 23:57:40 +03:00
Agatha Rose
63161444a9
Merge pull request #1 from otrapersona/dedup_trigger 
Add trigger to remove duplicate posts on db
 2021-06-04 22:58:42 +03:00
otrapersona
be8227c70a Changed group of trigger 
I think there's a tiny chance that two posts on diff instances have the same id, problem solved by using the uri.
 2021-03-13 13:54:32 -06:00
otrapersona
9f80c2746f Add trigger 
Fixes symptom but not cause 🤷‍♀️
 2021-03-13 13:46:18 -06:00
Agatha Rose
27f61c4374
Make bs4 only replace the tag name instead of name and contents 2021-02-18 18:01:43 +02:00
dependabot-preview[bot]
d07d49d42e
Merge pull request #43 from Lynnesbian/dependabot/pip/markovify-0.8.2 2020-08-02 05:04:11 +00:00
dependabot-preview[bot]
82943a1303
Bump markovify from 0.8.0 to 0.8.2 
Bumps [markovify](https://github.com/jsvine/markovify) from 0.8.0 to 0.8.2.
- [Release notes](https://github.com/jsvine/markovify/releases)
- [Commits](https://github.com/jsvine/markovify/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-08-02 05:03:05 +00:00
Lynne
09a1efc30a
Merge pull request #42 from Lynnesbian/dependabot/pip/requests-2.24.0 
Bump requests from 2.23.0 to 2.24.0
 2020-08-02 15:02:01 +10:00
Lynne
2baf060a08
Merge branch 'master' into dependabot/pip/requests-2.24.0 2020-08-02 15:01:49 +10:00
Lynne
64079a96cb
removed patreon 2020-08-02 14:54:22 +10:00
dependabot-preview[bot]
74046032aa
Bump requests from 2.23.0 to 2.24.0 
Bumps [requests](https://github.com/psf/requests) from 2.23.0 to 2.24.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/master/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.23.0...v2.24.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-08-02 04:25:23 +00:00
Lynne
bb39af52a9
Merge pull request #41 from Lynnesbian/dependabot/pip/beautifulsoup4-4.9.1 
Bump beautifulsoup4 from 4.8.2 to 4.9.1
 2020-08-02 14:24:33 +10:00
Lynne
96c047a40b
Merge pull request #39 from Lynnesbian/dependabot/pip/mastodon-py-1.5.1 
Bump mastodon-py from 1.5.0 to 1.5.1
 2020-08-02 14:24:14 +10:00
Lynne
8274409bf4
update extract code to match fedibooks 2020-05-27 22:31:16 +10:00
dependabot-preview[bot]
7b2fe14ba5
Bump beautifulsoup4 from 4.8.2 to 4.9.1 
Bumps [beautifulsoup4](http://www.crummy.com/software/BeautifulSoup/bs4/) from 4.8.2 to 4.9.1.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-05-18 19:13:06 +00:00
dependabot-preview[bot]
a5fd049309
Bump mastodon-py from 1.5.0 to 1.5.1 
Bumps [mastodon-py](https://github.com/halcy/Mastodon.py) from 1.5.0 to 1.5.1.
- [Release notes](https://github.com/halcy/Mastodon.py/releases)
- [Changelog](https://github.com/halcy/Mastodon.py/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/halcy/Mastodon.py/compare/1.5.0...1.5.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-03-16 19:16:47 +00:00
Lynnesbian
2321f75e11
added note about contacting me to help with the docker stuff 2020-03-11 18:14:59 +10:00