Using algoScraper in the Developer Tools Panel

TABLE OF CONTENTS

1. Overview

2. Understanding algoScraper Tool Window

2.1 Tool Functionality Summary

2.2 Scraping the Application Using algoScraper Tool

1. Overview

The algoscraper extension is designed to interact with web and mobile site applications and extract scraped data. However, due to security measures on some websites, you might face restrictions when using the extension directly. The following message is displayed if you are unable to scrape using the algoscraper extension:

To access algoScraper extension, perform the following:

1. Open the Developer Tools panel in your web browser by right-clicking on any part of the web page and selecting "Inspect" or by pressing F12 on your keyboard.
2. Once the Developer Tools panel is displayed, navigate to the "algoscraper" tab.
3. In the "algoScraper" window, you will find the capabilities provided by the extension. This extension enables you to interact with the web page and mobile browsers site, allowing you to extract the desired data.

Following sections describe the instructions of usage of the tool.

2.Understanding algoScraper Tool Window

Click the extension to view the algoScraper window

    2.1 Tool Functionality Summary

Field	Description
	Displays the Page name URL
	Click this button to scrape all the UI elements at once displayed on the particular page. When you scrape all the UI elements using this button, you can reset the page by clicking the Reset button. scrape the same page again  navigate to a different page within the application refresh the page. Note that regardless of the action you choose, your data will remain intact. All UI elements captured during previous actions will be displayed in table format without any loss of data. Additionally, you can use pagination at the end of the screen to navigate to all the UI elements captured.
	If you use invalid values in the search bar, you will see the error message shown in the left pane. Ensure that your search criteria are accurate and relevant to the data you are trying to retrieve. When you click the Reset button, search criteria are cleared.
	Click this button to reset all the scraped UI elements listed in the table format. A pop-up message is displayed to confirm the reset functionality.
	Click this button to access more actions menu.
Edit URL secondary window	Re-scrape Re-scraping involves re-extracting data from a web page. It is performed when changes occur on the web page or application under test. You can upload the updated re-scraped file while profiling the application and generate test cases. Choose this option to access the upload section and then perform the following actions: Upload the previously downloaded CSV file that contains the details of the scraped elements. Re-scrape the elements using this option and capture it in the same CSV file. Download the updated CSV file after re-scraping the elements. Upload the updated CSV file  Edit URL - choose this option to change the URL if needed. When you upload re-scraped file and click the Reset button, the URL defaults to the page URL that you are viewing.
	Select this button to add a new row to a scraped element table.
	Select this button to download the scraped UI elements table in a CSV format spreadsheet.
	Select this button if you wish to delete a particular row in a scraped UI element table.

2.2 Scraping the Application Using algoScraper Tool

Perform the following:

1. Open the application and if you wish to scrape the entire page at one go. Click Scrape UI button to scrape all the UI element details with corresponding XPath expression of the scraped UI page details will be listed in the table format.
   
   
2. If you wish to scrape individual UI element, click this icon to inspect a particular element. The captured details are added to the scraped element table list, as shown:
   
   
3. Common XPath refers to frequently used XPath expressions that are used to locate elements.
   If you wish to capture Common XPath expression, perform the following:
   Navigate to the particular element on a webpage, click this icon to inspect a common element, such as Back button, as shown: 
   
   Usage of common XPath expression: Common XPaths can be used across different web pages or applications to locate similar types of elements, such as buttons, input fields, or links.
   
   
4. JS Path helps testers extract specific data from JSON responses, such as retrieving names from a list of products: Products[*].name, where Products [*] name is a query.
   
   Let's consider an example where a user wants to search for a specific product by entering its name using a text input field and retrieving its details from a JSON response.
   
   In this example:
   • The user enters the name of a product they want to search for in the text input field.
   • Upon clicking the "Search" button, the searchProduct() function is called.
   • The function retrieves the user input, constructs a JSPath query to find the product details by name, and applies this query to the JSON response.
   • If a matching product is found, its details (name, price, and availability) are displayed in the UI. If no matching product is found, a message indicating that the product was not found is displayed.
     
     
5. After scraping UI elements, download the file in the CSV format. Upon successful download, a "File downloaded successfully" message is displayed, and click the icon to close the Scraper window.