Excel Web Query Secrets Revealed
Excel Web Query - What in the world is that? If you are like the other 99.9% of MS Excel users, you probably have never heard of microsoft excel web queries (note: statistic made up).
Excel web queries are powerful! Web queries are basically like having a web browser built into Excel that attempts to format the content, putting individual pieces of data into separate cells. You can then use Excel formulas (like =A1/B2) to work directly with the data you've downloaded. And you don't have to know anything about perl, cgi, php, javascript, etc.
Getting Started with Sample Excel Web Queries
The example web queries that come with Excel are usually for importing stock quote data into Excel. For more information about using web queries to download Stock Quotes into Excel, take a look at the Excel Stock Quotes Template.
Although importing stock quotes into Excel happened to be the reason I came across Excel web queries, I have since found many other ways to use them.
It is not the web query itself that is the secret that this article is about. Instead, it's the ability to make a DYNAMIC Web Query!
By dynamic, I'm not talking about using "refresh" to update the data. What I mean is the ability to change the parameters of the web query, such as typing in a new stock symbol into a cell and having the table update automatically.
If you find yourself using the internet to gather data by filling out forms and copying and pasting data over and over, an excel web query might be the solution to your monotonous woes. It's not always going to be better, and it's not always going to work, but it's worth a try, especially since a simple web query could boost your productivity!
Excel Web Query ".iqy" Files
The real key to creating a dynamic excel web query is to create your own ".iqy" file. In it's basic form, the ".iqy" file is simply a TEXT file with three main lines:
WEB 1 https://www.thedomain.com/script.pl?paramname=value¶m2=value2
You can create the file using a simple text editor! Most text editors will save a new file with the .txt extension, but you can change the extension to .iqy after you create your text file. If it is already a .iqy file and you want to edit it, you may need to right-click on it to select NotePad or WordPad (or some other text editor) to open it.
The third line is the important one! It's simply the URL or web page address. Notice that it contains two parameters. If you don't know what parameters are for, just browse the web for a while and pay attention to what shows up in the address bar of your browser. Go to Google.com for instance, and look up "excel web query".
Parameter name / value pairs are listed after the "?" in the URL and are separated by an "&".
Make the Web Query Dynamic
To make the query dynamic, replace the value of each parameter in the web query file (queryname.iqy) with:
["paramname","Enter the value for paramname:"]
Want to see how this would apply to a Google search? The form that I used above consists of HTML code that looks like this:
<form action="https://www.google.com/search">
<input type="text" name="q" value="excel web query">
<input type="submit" value="Search Google">
</form>
Notice that "q" is the name of the parameter, and the action tells you what the URL should be. The dynamic web query file for a simple google search would look like this:
WEB 1 https://www.google.com/search?q=["keyword","Enter the Search Term:"]
Let's Create a Web Query
- Open up a text editor and copy the Google example. Save the file as GoogleSearch.iqy
- Open up Excel and enter a search term in Cell B3
- Open the web query file you just saved (GoogleSearch.iqy) by double-clicking on the file, or open Excel and go to Data > Existing Connections > Browse for More.
- When prompted for the search term, enter =B3. If that doesn't work, just enter a search term for now.
Play around with the web query options and properties by right-clicking within the query results and selecting Parameters or Data Range Properties or Edit Query.
When you go to Edit Query, you can choose to select a specific portion or table within the page or the entire page (by clicking on one of the yellow arrow boxes in the preview). Then, you can save your new query as a .iqy file and look at what parameters it uses.
WEB 1 https://www.google.com/search?q=["q","Enter the Search Term:"] Selection=EntirePage Formatting=RTF PreFormattedTextToColumns=True ConsecutiveDelimitersAsOne=True SingleBlockTextImport=False DisableDateRecognition=False DisableRedirections=True
When you have the query looking just the way you want it, save the Excel web query (an icon or button for saving the web query is in the "Edit Query" window. When you save the new query, your ".iqy" file will include the options you have selected.
Other Example Web Queries
Below are a few examples of .iqy files that you might try. Websites often change, or they may change the way their APIs work, so these examples might not work forever.
Example 1: Get historical daily stock prices from Investopedia. Note that the start and end dates are text, so if you are setting up date parameters in Excel, the cells you link to need to be text rather than date values. This query uses 3 parameters: Symbol, StartDate, EndDate.
See this article for other sources for historical stock quotes.
Example 2: Get data from a published Google Spreadsheet. See my article Excel Stock Quotes in Excel to see how you can publish a Google Spreadsheet and then use a web query to get the data into Excel.
Become a Web Query Wiz
Take a few minutes to think about what mundane or repetitive tasks YOU do on the internet, particularly cases where you go to the same sites over and over to get data. If you find yourself copying and pasting or re-typing information into Excel so that you can perform calculations with the data, you may have found an ideal use for an excel web query.
There are two main things that you need in order to become a Web Query Wiz.
- A working knowledge of Excel formulas. By this, I mean the ability to use text-manipulation and other formulas. A GREAT book for both learning and reference (and the one that I use all the time) is John Walkenbach's "Excel Formulas". If you are an avid Excel user, it will probably be the best investment you will make for a long time! The only problem I have with the book is that I need TWO copies - one for work and one for home.
- A working knowledge of HTML. This is the hard part unless you are a web designer or programmer. The articles listed in the side bar will help to some extent, but ultimately this issue was the reason why I didn't write a more detailed tutorial. Something that will be a great help is to learn about how <form> tags work. Here is my favorite reference.
If you found this article interesting, be sure to check out some of my other Excel tips.
- Jon Wittwer, President
TotalSheets, LLC