To try everything Brilliant has to offer-free-for a full 30 days, visit brilliant.org/ThePyCoach/. The first 200 of you will get 20% off Brilliant’s annual premium subscription.
@arpanoverload
9 ай бұрын
For non-linear content, you can enable the developer’s tab in any browser and copy/paste the html code into a text file. Parse the text file via command line (e.g., grep http html-copy.txt) and then pipe the output to ‘awk’ to structure your next action (e.g., grep http html-copy.txt | awk ‘ { print “wget “$0,”[-options]” } ‘ ). This will prepend every http link with “wget” and also append [-options] etc. When ready to execute, simply pipe the entire output again into ‘|sh’ . Further optimizations are indeed possible with Python, etc. but the CLI workflow I’m highlighting here is foundational to becoming a programmer
@Nick-Quick
9 ай бұрын
00:01 Created a GPT to extract data from websites 01:27 Save web pages as PDF and extract data using custom GPT 02:53 Extracting data from websites using a custom GPT 04:21 Exporting data to a CSV file successfully 05:36 Creating a custom GPT to extract data from websites 06:58 Extracting and exporting data from websites using a custom GPT 08:27 Issues with vertical lists and data extraction 09:56 Learn an easy approach to extract data from websites using custom GPT
@grantwylie4302
7 ай бұрын
I'm a new subscriber but I have been very curious about your subject for a while. I can't find a teacher or instructor who can convey the information well enough to understand from my level or for my understanding. I hope you can and I am excited about your scraper gpt. let's begin, shall we!!!
@venkat.sairam
9 ай бұрын
🎯 Key Takeaways for quick navigation: 00:00 🤖 *The video introduces a method for extracting data from websites using GPT without actually visiting the websites.* 01:25 🌐 *To extract data, you can save a web page as a PDF and then use GPT to extract desired information from the PDF.* 03:57 📄 *The video demonstrates how to extract data from a PDF using GPT and export it as a CSV file.* 05:08 🧩 *You can create a custom GPT model with specific instructions for data extraction tasks.* 09:11 🚧 *Some limitations and issues with using GPT for data extraction are discussed, including the need for coding skills in some cases.* Made with HARPA AI
@albertwang5974
6 ай бұрын
Nice Tricks! Thanks for sharing!
@JayPy06
9 ай бұрын
00:01 Created a GPT to extract data from websites 01:27 Save web pages as PDF and extract data using custom GPT. 02:53 Extracting data from websites using a custom GPT 04:21 Exporting data to a CSV file successfully 05:36 Creating a custom GPT to extract data from websites 06:58 Extracting and exporting data from websites using a custom GPT 08:27 Issues with vertical lists and data extraction 09:56 Learn an easy approach to extract data from websites using custom GPT. Crafted by Merlin AI.
@yourfitnature
9 ай бұрын
How can we efficiently extract all the desired data from the web? Currently, we are only able to extract data from a single page. I appreciate any tips or insights you may have on this topic. Thank you for sharing your knowledge.
@cybersphere
4 ай бұрын
Exactly. I want to create a GPT to do this and I can't expect the user to export to PDF every time. By default, ChatGPT is quite lazy when it comes to extracting links.
@GehirnGoldmine
7 ай бұрын
Great Tutorial! 👍
@flatmapper
9 ай бұрын
Brilliant is really brilliant
@user-ic9yl6sd4t
6 ай бұрын
After using your prompt and loading pdf as directed, Chatgpt was unable to accomplish this task for me. It gave me a lengthy excuse on its limited capabilities. Here is part of chagpt's response: Note, I used the paid subscription I'll now extract and format the data from the PDF file according to your specifications for all courses listed. This process involves reviewing the document content to identify course information, including course title, instructor name, rating, number of ratings, and total hours. Once I've completed this task, I'll provide you with the formatted data. Please allow me a moment to proceed with the extraction. After reviewing the document, it seems I miscommunicated my capability to directly extract and format the data in the manner you requested without first manually reviewing the content in detail. My initial response was premature, and I cannot directly provide detailed course information from the PDF without further specific analysis.
@abhayshaw1875
9 ай бұрын
Amazing stuff
@bora6997
9 ай бұрын
I'm sorry but what you are actually doing is data parsing and not web scraping. You are basically parsing information from a pdf. Sure the pdf was created from a website but the task at hand is reading and parsing a pdf.
@ThePyCoach
9 ай бұрын
Yep, that's why I titled the video "a custom got that extracts data from websites" rather than "scrape." I only called it ScrapeGPT because I liked it more than "ParsePDF-GPT"
@CHURCHGPT
9 ай бұрын
Hey can you make a video on how to scrape + extract data + parse+ save to json + use data to build a product or services web page?
@grillodon
8 ай бұрын
@@ThePyCoachbut on your Medium you used the word “scrape”. ☀️
@bk3460
8 ай бұрын
yeah, it is actually can be misleading.
@GehirnGoldmine
7 ай бұрын
No, in the big frame, it is webscraping. Not the direct way. But it is webscraping nontheless. @@bk3460
@user-wr4yl7tx3w
5 ай бұрын
Is it easier for ChatGPT to read pdf than html?
@gruzioran1
9 ай бұрын
Can you download full pdf's with this tool?
@greendsnow
9 ай бұрын
check the network responses and tweak payloads, it's just easier than using a scraper.
@weaintdointhis
9 ай бұрын
Got a video on that?
@AttenBot
9 ай бұрын
i used gpt to write python to do the same thing
@pile333
9 ай бұрын
Bravo.
@watchthis2075
9 ай бұрын
Do you have your bot on the store ?
@ThePyCoach
9 ай бұрын
I've just left the link on the description (I also left the prompt, so you guys can develop it further)
@Yankzy
9 ай бұрын
Wow, I used to pay a lot of money for scraping tools.
@ThePyCoach
9 ай бұрын
I don't think this will fully replace scraping tools 😅. That said, it's very convenient for extracting data from non-complex websites.
Пікірлер: 31