{"id":312,"date":"2013-09-11T01:01:57","date_gmt":"2013-09-11T01:01:57","guid":{"rendered":"http:\/\/seogadget.com\/?p=14503"},"modified":"2014-01-19T15:59:47","modified_gmt":"2014-01-19T15:59:47","slug":"scraping-schema-markup-for-competitive-intelligence","status":"publish","type":"post","link":"https:\/\/www.urlrate.com\/blog\/scraping-schema-markup-for-competitive-intelligence\/","title":{"rendered":"Scraping Schema Markup for Competitive Intelligence"},"content":{"rendered":"<p>Structured mark up is an important for e-commerce websites in the event that they want to stand out in the SERPs. Because e-commerce websites are usually set as much as scale, scraping all of their knowledge is so easy. All it takes is a Screaming Frog crawl and Outwit Hub.<\/p>\n<p>For dropshippers and affiliate websites, harvesting competitor knowledge within schema mark up tags can be extraordinarily helpful. In case you are promoting the identical products as your competitors, that you would be able to evaluate pricing, product descriptions, calls to motion\/unique promotions &#8211; anything else &#8211; and analyze how you stack up towards your rivals.<\/p>\n<p>Before we are able to begin, we want to figure out where products continue to exist the competitor web site. In case your competitor has naturally built out data architecture, it shouldn&#8217;t be too tough. On Target.com, they use the directory \u00a0\/p\/ for their merchandise.<\/p>\n<h3 style=\"text-align: center\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter  wp-image-14513\" alt=\"target_IA_example\" src=\"http:\/\/seogadget.com\/wp-content\/uploads\/2013\/09\/target_IA_example.png\" width=\"527\" height=\"402\" \/><\/h3>\n<h2>Step 1) Crawl and Gather Product Pages<\/h2>\n<p>So as to get the pages that are living underneath the \/p\/ listing, fan the flames of <a href=\"http:\/\/www.screamingfrog.co.uk\/seo-spider\/\">Screaming Frog<\/a> and beneath Configuration &gt; Include, \u00a0add .*\/p\/.*<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-14514\" alt=\"include p directory to snag products\" src=\"http:\/\/seogadget.com\/wp-content\/uploads\/2013\/09\/include_for_products_screaming_frog.png\" width=\"414\" height=\"347\" \/><\/p>\n<p style=\"text-align: center\"><strong>Now your Screaming Frog export will handiest include product pages<\/strong><\/p>\n<p style=\"text-align: left\">So everyone can observe alongside and work with the same information, I&#8217;ve long gone ahead and scraped all of the laptops which can be presently listed on the Target.com site, which you could get right here:<\/p>\n<p style=\"text-align: center\"><a href=\"https:\/\/docs.google.com\/spreadsheet\/pub?key=0AqOQkR7YOK3XdFUyN1lmdFBQZW5kejI2bi1hQkVFTVE&amp;single=true&amp;gid=0&amp;output=html\">\u00a0Listing of Goal Laptops (09\/10\/2013)<\/a><\/p>\n<h2 style=\"text-align: left\">Step 2) Analyze Structured Markup and On Page Components<\/h2>\n<p>Take one of the most product pages from your Screaming Frog Export, for this case, we&#8217;ll use the\u00a0<a href=\"http:\/\/www.target.com\/p\/acer-aspire-11-6-touch-screen-laptop-pc-s7-191-6640-us-with-128gb-ssd-4gb-memory-silver\/-\/A-14292747#prodSlot=large_1_30\">Acer Aspire 11.6&#8243; Contact Display Computer PC<\/a>\u00a0page. Should you <a href=\"http:\/\/www.google.com\/webmasters\/tools\/richsnippets?q=http%3A%2F%2Fwww.target.com%2Fp%2Facer-aspire-11-6-touch-screen-laptop-pc-s7-191-6640-us-with-128gb-ssd-4gb-memory-silver%2F-%2FA-14292747%23prodSlot%3Dlarge_1_30\">enter the URL into the Rich Snippet Testing Software<\/a> you can find that Goal is using a ton of structured markup on their product pages.<\/p>\n<p><strong>For this, exercise, we&#8217;re going to scrape:<\/strong><\/p>\n<ul>\n<li><strong>Price<\/strong><\/li>\n<li><strong>SKU<\/strong><\/li>\n<li><strong>Product Identify<\/strong><\/li>\n<li><strong>Battery Cost Life (non-schema part)<\/strong><\/li>\n<li><strong>Name to motion\/Promoting (non-schema component)<\/strong><\/li>\n<\/ul>\n<h2>Step 3) Stir up OutWit Hub<\/h2>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-14536\" alt=\"Outwit Hub Logo\" src=\"http:\/\/seogadget.com\/wp-content\/uploads\/2013\/09\/Outwit-Hub-Logo.gif\" width=\"155\" height=\"163\" \/><\/p>\n<p><a href=\"http:\/\/www.outwit.com\/\">Outwit Hub<\/a> is a personal computer scraper\/information harvester. It costs $60 a yr and is neatly value it. Outwit can utilize cookies, so scraping behind a pay-wall or password safe website online is a non-problem.\u00a0As an alternative of getting to make use of Xpath to scrape information, Outwit Hub allows you to highlight the supply code and set markers to scrape the whole lot that lies in between. If you are no longer a technical marketer, and you find yourself having to assemble a variety of data\/wasting your time &#8211; this is a good software to have for your arsenal.<\/p>\n<h2>\u00a0Step 4) Build Your Scraper<\/h2>\n<p><em>This can be intimidating in the beginning, but it surely&#8217;s a lot extra scalable then trying to use Excel or Google medical Doctors to scrape hundreds and hundreds of data points<\/em><\/p>\n<p>In the proper-hand menu, click on on Scrapers. Enter the instance Target URL. This may load the supply code.<\/p>\n<p>Click on the &#8220;New&#8221; Button on the lower component to the monitor and name your scraper. I&#8217;m calling mine, &#8220;<strong>Goal Computer Scraper<\/strong>.&#8221;<\/p>\n<p style=\"text-align: center\"><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-14516  aligncenter\" alt=\"Outwit_Scraper_Build\" src=\"http:\/\/seogadget.com\/wp-content\/uploads\/2013\/09\/Outwit_Scraper_Build.png\" width=\"542\" height=\"349\" \/><\/p>\n<p style=\"text-align: left\">withIn the search field, start entering within the markup for the schema tags you wish to have to scrape for. Remember this isn&#8217;t Xpath, you don&#8217;t wish to worry in regards to the DOM, you handiest want to figure out what distinctive supply code goes prior to the part (the schema tag) and what&#8217;s after it.<\/p>\n<h2 style=\"text-align: center\">Extreme Close Up!<\/h2>\n<p style=\"text-align: center\"><a href=\"http:\/\/seogadget.com\/wp-content\/uploads\/2013\/09\/Scraper_Build_Close_Up.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter  wp-image-14519\" alt=\"Scraper_Build_Close_Up\" src=\"http:\/\/seogadget.com\/wp-content\/uploads\/2013\/09\/Scraper_Build_Close_Up.png\" width=\"685\" height=\"177\" \/><\/a><\/p>\n<p>It&#8217;ll take some practice at first, however once you get the cling of it, it is going to most effective take a few minutes to set up a customized scraper.<\/p>\n<h2>Step 5) Test Your Scraper<\/h2>\n<p>While you&#8217;re executed entering in the markers for the data you want to assemble, hit the execute button and test your outcomes. You must see one thing like this:<\/p>\n<p style=\"text-align: center\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter  wp-image-14525\" alt=\"scraper_test_for_outwit_hub\" src=\"http:\/\/seogadget.com\/wp-content\/uploads\/2013\/09\/scraper_test_for_outwit_hub.png\" width=\"714\" height=\"102\" \/><\/p>\n<p>&nbsp;<\/p>\n<h2>\u00a0Step 6) Put the checklist of URLs right into a .txt file and save it.<\/h2>\n<div id=\"attachment_14543\" style=\"width: 510px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-14543\" alt=\"disks for saving\" src=\"http:\/\/seogadget.com\/wp-content\/uploads\/2013\/09\/disks-for-saving.jpg\" width=\"500\" height=\"333\" \/><\/p>\n<p class=\"wp-caption-text\">Any of those storage units or your local computing device will do<\/p>\n<\/div>\n<h2>\u00a0Step 7) Open the .txt file in Outwit using the file menu<\/h2>\n<p>If you happen to go to the left navigation, just below the main listing, there&#8217;s a subdirectory known as &#8220;Links.&#8221; Click on on Links in the left-hand nav. This is what you should see:<\/p>\n<p style=\"text-align: center\"><a href=\"http:\/\/seogadget.com\/wp-content\/uploads\/2013\/09\/tons_of_links_from_outwit.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter  wp-image-14528\" alt=\"a list of links from outwit to scrape\" src=\"http:\/\/seogadget.com\/wp-content\/uploads\/2013\/09\/tons_of_links_from_outwit.png\" width=\"523\" height=\"351\" \/><\/a><\/p>\n<p style=\"text-align: left\">Choose the entire knowledge using Regulate+A and then proper click on the row with the entire URLs.<\/p>\n<h2>\u00a0Step 7) Fast Scrape!<\/h2>\n<p style=\"text-align: center\"><a href=\"http:\/\/seogadget.com\/wp-content\/uploads\/2013\/09\/Scraping_tons_of_links_with_outwit.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter  wp-image-14527\" alt=\"scraping tons of schema with outwit\" src=\"http:\/\/seogadget.com\/wp-content\/uploads\/2013\/09\/Scraping_tons_of_links_with_outwit.png\" width=\"546\" height=\"485\" \/><\/a><\/p>\n<p>In the right click on menu, make a choice:\u00a0Auto-Explore &gt;Quick Scrape (Embrace Selected Knowledge) &gt; And make a choice the scraper we just built collectively.<\/p>\n<p style=\"text-align: center\"><a href=\"http:\/\/screencast.com\/t\/ntOZZzlYd38a\">right Here&#8217;s a video of the closing step in Outwit<\/a><\/p>\n<h2>Step eight) Bask within the glory of your competitor&#8217;s knowledge<\/h2>\n<p style=\"text-align: center\">\u00a0<img loading=\"lazy\" decoding=\"async\" class=\"wp-image-14539\" alt=\"scraped pricing data from target using outwit\" src=\"http:\/\/seogadget.com\/wp-content\/uploads\/2013\/09\/end_product_-_scraped_pricing_data_from_target.png\" width=\"637\" height=\"383\" \/><\/p>\n<p style=\"text-align: left\">withIn the left-hand navigation, there is a class referred to as &#8220;knowledge&#8221;, with the subcategory &#8220;scraped&#8221; &#8211; simply if you happen to navigated faraway from it, that&#8217;s the place your entire information might be saved, simply be careful to not load a brand new URL in Outwit Hub or else it&#8217;s going to be written over and you&#8217;ll have to scrape all over once more.<\/p>\n<p style=\"text-align: left\">You could export your data into HTML, TXT, CSV, SQL or Excel. I most often simply go for an Excel export and do a <a href=\"http:\/\/seogadget.com\/using-vlookup\/\">VLOOKUP<\/a> to combine the information with the unique Screaming Frog crawl from the 1st step in Excel.<\/p>\n<h1 style=\"text-align: center\">Bought any fun doable use cases?<\/h1>\n<h1 style=\"text-align: center\">Share them beneath in the feedback!<\/h1>\n<pre style=\"text-align: left\">Image source by means of <a href=\"http:\/\/www.flickr.com\/photos\/avaragado\/\">Flickr consumer avargado<\/a><\/pre>\n<p>The post <a href=\"http:\/\/seogadget.com\/scraping-schema\/\">Scraping Schema Markup for Aggressive Intelligence<\/a> regarded first on <a href=\"http:\/\/seogadget.com\/\">SEOgadget<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Structured mark up is crucial for e-commerce websites if they want to stand out in the SERPs. Because e-commerce sites are generally set up to scale, scraping all of their information is very easy. All it takes is a Screaming Frog crawl and Outwit Hub. For dropshippers and affiliate sites, harvesting competitor data within schema [&hellip;]<\/p>\n<p>The post <a href=\"http:\/\/seogadget.com\/scraping-schema\/\">Scraping Schema Markup for Competitive Intelligence<\/a> appeared first on <a href=\"http:\/\/seogadget.com\/\">SEOgadget<\/a>.<\/p>\n","protected":false},"author":8,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[70,71],"class_list":["post-312","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-guides","tag-microformats-html5"],"_links":{"self":[{"href":"https:\/\/www.urlrate.com\/blog\/wp-json\/wp\/v2\/posts\/312","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.urlrate.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.urlrate.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.urlrate.com\/blog\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/www.urlrate.com\/blog\/wp-json\/wp\/v2\/comments?post=312"}],"version-history":[{"count":4,"href":"https:\/\/www.urlrate.com\/blog\/wp-json\/wp\/v2\/posts\/312\/revisions"}],"predecessor-version":[{"id":572,"href":"https:\/\/www.urlrate.com\/blog\/wp-json\/wp\/v2\/posts\/312\/revisions\/572"}],"wp:attachment":[{"href":"https:\/\/www.urlrate.com\/blog\/wp-json\/wp\/v2\/media?parent=312"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.urlrate.com\/blog\/wp-json\/wp\/v2\/categories?post=312"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.urlrate.com\/blog\/wp-json\/wp\/v2\/tags?post=312"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}