Skip to content

Replace abandoned random-user-agent with fake-useragent #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nagapraneethk wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Replace abandoned random-user-agent with fake-useragent #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ The default `dict.txt` file was taken from from [Conf-Thief](https://raw.githubu

* Search for provided keywords
* Handle rate limiting by itself, as long as the returned status code equals `HTTP 429`, or specify max requests per second in CLI
* The user agent is randomized
* The user agent is randomized using fake-useragent
* Proxying is supported either via HTTP or socks. See cli help for examples
* Custom CQL
* SSL/TLS checks are disabled by default
Expand Down
2 changes: 1 addition & 1 deletion requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@ atlassian-python-api==3.41.3
beautifulsoup4==4.12.2
requests-ratelimiter==0.4.2
clize==5.0.2
random-user-agent==1.0.1
fake-useragent
readchar==0.7
PySocks==1.7.1
2 changes: 1 addition & 1 deletion setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
"beautifulsoup4>=4",
"requests-ratelimiter",
"clize>=5",
"random-user-agent>=1",
"fake-useragent",
"readchar",
"PySocks>=1",
],
Expand Down
7 changes: 2 additions & 5 deletions src/conkeyscan/conkeyscan.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,7 @@
from bs4 import BeautifulSoup
from requests_ratelimiter import LimiterSession
from requests.packages.urllib3.exceptions import InsecureRequestWarning
from random_user_agent.user_agent import UserAgent
from random_user_agent.params import SoftwareName, OperatingSystem
from fake_useragent import UserAgent

# disable annoying insecure requests warnings
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
Expand Down Expand Up @@ -168,9 +167,7 @@ def main(
rate_limited_session = LimiterSession(per_second=rate_limit)

if not user_agent:
user_agent = UserAgent(
software_names=SoftwareName.CHROME.value, limit=100
).get_random_user_agent()
user_agent = UserAgent().random
logger.debug("Using User Agent: " + user_agent)
rate_limited_session.headers.update({"User-Agent": user_agent})

Expand Down