admin-valentin/openai-cookbook
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Revert "File name sanitization (#630)" (#668)

Browse source 
This reverts commit 169f5e02c8.
This commit is contained in:
Simón Fishman 2023-08-29 17:45:47 -07:00 committed by GitHub
parent fae14ddb89
commit b2ca4d395c
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
apps/web-crawl-q-and-a/web-qa.ipynb
View file

@ -628,11 +628,8 @@
" url = queue.pop()\n", " url = queue.pop()\n",
" print(url) # for debugging and to see the progress\n", " print(url) # for debugging and to see the progress\n",
"\n", "\n",
" sanitized_url = re.sub(r'[^\w\s.-]', '_', url)\n",
" file_path = 'text/' + local_domain + '/' + sanitized_url[8:] + \".txt\"\n",
"\n",
" # Save text from the url to a <url>.txt file\n", " # Save text from the url to a <url>.txt file\n",
" with open(file_path, \"w\") as f:\n", " with open('text/'+local_domain+'/'+url[8:].replace(\"/\", \"_\") + \".txt\", \"w\") as f:\n",
"\n", "\n",
" # Get the text from the URL using BeautifulSoup\n", " # Get the text from the URL using BeautifulSoup\n",
" soup = BeautifulSoup(requests.get(url).text, \"html.parser\")\n", " soup = BeautifulSoup(requests.get(url).text, \"html.parser\")\n",