Data Scrape From Webpage
Jul 7, 2014
I am currently working on a VBA Macro in order to scrape data from a webpage into an Excel spreadsheet. My problem originates from the type of webpage that I am accessing. I am accessing a webpage that is used to filter the type of information, in which I have a Favorite (preset), which is the URL that I have included in the code. Once I get to that page, the code selects the proper function to load the data onto another webpage. It is from this webpage that the data I need to import into the spreadsheet is located.
Here is a step by step breakdown of my macro and the bolded/underlined portion .
Macro initiated - > filter webpage opened - > selection of filter preloaded - > data loaded into another webpage - > webpage with data opened - > | need to transfer data to Excel |
View 4 Replies
ADVERTISEMENT
Aug 11, 2009
basically you put a webpage addresses in column A and it imports the webapages and then uses tables "1" and "3" to cut data into a new sheet.
it runs until it gets to the end of column A and so has scraped all webpage addresses i have entered into column A.
unforunately now it does a couple (or sometimes none) then crashes.
runtime error 1004 - something about a file not being there?
Sub Macro2()
a = 1
Sheets(3).Select
While Sheets(1).Cells(a, 1) ""
urladdress = "URL;" & Sheets(1).Cells(a, 1).Text
View 9 Replies
View Related
Apr 17, 2012
I need to scrape data from excel cells. I have data such as "##aaa###". I need to get the numbers that preceed the letters. There may be 2 or 3 or more numbers prior to the first letter so it needs to be a little flexible.
View 1 Replies
View Related
Mar 14, 2014
I am trying to open/save a csv file from a website but the URL only takes me to the page the file is hosted on then the file has to be clicked and it asks you to save it. It doesn't appear that it is a shortened URL and using Firebug I have gone through the HTML trying to find a link to the file but haven't. These files are date specific and ultimately the macro will be used to scrape different dates.
The URL: [URL] From that you will select the year>month>day number>download file
View 5 Replies
View Related
Oct 23, 2011
have been trying to work out how to copy data from a website into a worksheet. For some reason the Data > Import External Data > New Web Query route does not work so I have tried the code below but cannot get this to work either.
What I am trying to achieve is to copy each of the various tables on the page into Excel.
Dim WebCopy As Object
Dim WebtitleCopy As Object
Set WebCopy = Sheets("Sheet2")
WebCopy.Cells.Clear
[Code].....
View 8 Replies
View Related
Dec 14, 2011
From the website [URL] ....
I need the Name, Website and contact details. I tried to extract but there is no ID for the element.
Data from the above site is:
1 Name: Shopping da Midia
2. Website: It is a hyperlink of Shopping da Midia that is [URL] ......
3. Contact details:
Tel: 16 2101-2500
Av. Independencia , 192
CEP: 14010-210
Ribeireo Preto
E-mail: compras@cdvirgem.com.br
View 5 Replies
View Related
Apr 2, 2009
Fetch data from the table on the page given by the link below..
http://www.moneycontrol.com/stocks/f...l&sort_order=0
View 9 Replies
View Related
Jun 29, 2009
If I copy the source to the notepad, I can paste it to the spreadsheet IF I use the "PASTE" and then click on the "Use Text Import Wizard". This works OK, but, I am needing to do this from INSIDE another macro, and I can not rely on the users, so I need this to be automated.
I tried to RECORD the steps, but no matter what, it records only:
Sub Macro1
Range("C1").Select
ActiveSheet.PasteSpecial Format:="Unicode Text", Link:=False, _
DisplayAsIcon:=False
End Sub
But does NOT include the selections I made with the wizard (delimited with "`") or FIXEDWIDTH. Eitherway works fine, but the macro won't.
Attached is a sample of the source.
Saving the Notepad, and then processing the FILE is not an option. I need to work straight from the clipboard.
I have spent hours looking for the different options available for the "PasteSpecial" but all I find is more people asking what the options are.
File import, I found out how to control the data, same with convert to text. But it does not seem to be "quite" the same
View 9 Replies
View Related
Jan 31, 2007
I am having trouble pulling data from a website. I need to get data from this website:
{url}
I need the highs and the low temperatures for the next week. (just the numbers preferably). But when I go to Data>> Import External Data >> New Web Query i'm not able to get any of the numbers into Excel. This way worked fine with other websites.
View 3 Replies
View Related
Sep 12, 2013
I tried to use the following code to get data from an ASPX webpage:
[URL]....
VB:
Sub Fetch_Marcas()
Dim IeTbl As Object
Dim obj As Object
Dim Scrn As String
Dim objIE As SHDocVw.InternetExplorer 'Microsoft Internet controls (shdocvw.dll)
[Code]....
However, I cannot get past the point where I click over the register (highlighted in yellow in Picture 1), to get the data displayed at Picture 2.
Picture 1
Picture 1.jpg
Picture 2
Picture 2.jpg
Please note that this query requires to click over what is not a button (I looked in the forum, but no similar question was made before on this topic).
View 6 Replies
View Related
Jul 28, 2006
How do I send data via VBA from a spreadsheet to a webpage that is already open? I don't want to open another IE window. I just want to be able to navigate the IE window that is already open on my desktop. Also, in reference to the open webpage, with this particular webpage (secure site), when you enter data on 1 screen, it takes you to another screen with a different url where you have to enter a bit more data and then move to a third screen with a different url and enter the rest of the data and then submit. Is there a way in VBA to have one macro handle passing data to all 3 screens and submit the information?
View 4 Replies
View Related
Jan 23, 2008
At work we have a particular website open at all times. It pulls random data that is entered by another office and we then pull it up using a unique reference number generated. Is there anyway to create a macro that will scrape the data from this open page? I know that you can use the AppActivate command to set the focus on this particular page, but is there a way to get the data I need from there? I know you can bring up a new webpage with Set IE, but is there a way to use the Set IE with whatever page currently has the focus? Basically my project is if we see any errors, we have to pull up a form I made and enter the data manually. I am just hoping there is a way to scrape the basic information each time so we don't have to do it all manually.
View 4 Replies
View Related
Jul 15, 2014
I'm trying to extract some data from an online page, but I require a specific cell at an exact time, each day.
For example, I would like to acquire the main data from the following page: [URL]
into excel, but additionally, I require the ESU14 (Sep '14) Open cell at exactly 0700 BST. I am currently using the 'Data' - 'From Web' feature to scrape the necessary fields into Excel however I haven't found a way to acquire one of those cells, at exactly a certain time, every day, even when Excel et al aren't open.
View 1 Replies
View Related
Jul 31, 2005
I am trying to import data from a website protected by ID and password. I do have the login ID and Password to login to the website.
I am importing data from the website to my excel sheet to track some shipping informations. I do have the 'Import to Excel' option but I would rather open the
spreadsheet and click Refresh Data and have all the Data imported with click of a button.
I have already tried to use Import External Data but it is not working.
View 9 Replies
View Related
May 13, 2008
I am having a really difficult time with this code. I am trying to fill in some fields on an webpage in an effort to eventually retrieve some data. I have read lots of posts on the topic and somehow I'm still confused. My code crashes on this line and gives an Automation error, Unspecified Error message: With appIE.document.forms(0). No matter what I try, I can't get past this line. I have also tried: With appIE.document.all
Sub PropInfo()
Dim appIE As SHDocVw.InternetExplorer
Set appIE = CreateObject("INTERNETEXPLORER.APPLICATION")
appIE.Visible = True
appIE.navigate "http://gisims2.miamidade.gov/MyHome/proptext.asp"
Do While appIE.Busy: DoEvents: Loop
Do While appIE.readyState <> complete: DoEvents: Loop
With appIE.document.all
.Item("cmd").Value = "FINDADDR"
.Item("cmdTemp").Value = "FINDADDR"
.Item("searchtool").Value = "ADDR" 'Search by' dropdown
.Item("stnum").Value = "2417" 'house #' field
.Item("stdir").Value = ""
.Item("stname").Value = "ponce de leon" 'street name' field
.Item("sttype").Value = "BLVD" 'street type' field
.submit
End With
End Sub
View 2 Replies
View Related
Sep 1, 2007
I recived the small job of making a template for my company for travel. I had spare time so I added in a feature so they can just e-mail it to the approite person from clicking a command button intead of taking the time to click the built in excel e-mail function and typing the addresse.
Anyway, this from tells the person who it is being e-mail to, what flights to book.
I have differnt command buttons to bring up differnt airline webpages, bascially a hyperlink but in command button form. Is there a way, just because i have time, to take the flying dates from excel, input them into the webpage which the command brings up, and search for the flights for that date, just by clicking a button in excel?
I am using excel VB for the little code I have
View 9 Replies
View Related
Dec 14, 2009
I need a macro which opens a webpage, fills the search box with given value(from cell A1) and click the search button.
The macro I have now, opens the page. I need the rest. The webpage is something similar to google homepage.
View 9 Replies
View Related
Jan 22, 2014
I was reading Box Office Mojo and wanted to know which actor would have the highest box office total if worldwide sales were accounted for and inflation was considered (for some reason they offer the data in 3 different formats but that is not one of them). I had previously only used Excel for simple records so I had to search your site for ways to write a few macros but I got everything working. I want to present the data.
It looks like embedding a spreadsheet in a webpage can be done but is there anyway I can add a feature so that if the person browsing the site clicks on a cell that cell's data will removed from consideration? I don't want the cell to actually display a "0" but if the background color could change to signify it was no longer being counted that would awesome. A lot of people argue about what movie should be counted toward an actors total box office record (starring roles only, supporting roles, voice work, cameos) so I would really like to be able to allow people to easily remove roles they personally don't think should count towards the total.
View 1 Replies
View Related
Apr 8, 2014
I want to write a macro with which i can click on a particular hyperlink in that website.
For example...the website is [URL]...In this websote I want to click on the particular country (Let say "Gabon")...but I want to click it by using vba from excel.
View 4 Replies
View Related
Jul 22, 2010
I have a simple text box I need to fill in with my email which I have managed to do very simply with this code;
Code:
Dim IE As InternetExplorer
Dim sIEWindowName As String
Set IE = New InternetExplorer
IE.Visible = True
[Code]...
The layout looks like this with my email.value entered in.
I'm using Access 2007, but that is irrelevant for the VBA.
My problem is I can't enter the filename as easily. I have to use SENDKEYS to TAB over then a bunch of other tacky SENDKEY commands to load my file. Even though it works I don't feel it's reliable and would rather just fill the file name in a bit easier.
I used IE developer to show before and after using the BROWSE button to load the file location as seen in pic 1 and pic 2.
The code below has been my best attempt. It runs without error but it does not give me the result I need which is a file location for the application to pull.
Code:
IE.Document.form1.FileName.Value = "C:New FolderaFileToUpload.txt"
View 5 Replies
View Related
Jan 7, 2012
Copy the contents of a text file from a webpage?
For instance, from this page I only want the data in the text box, which can be selected by clicking the Highlight All button.
I've seen code to copy an entire page, but this does not capture the text box contents in this case.
View 9 Replies
View Related
Jan 15, 2013
I have a number of macros which i use to open up a web page in IE, copy the data and then paste it into excel. however i have a problem which i cant solve.
The issue is that the web page is a link to a pdf. So while i can open the page i cant seem to copy the data.
Note that the data that i am trying to copy out of the pdf is a table of data
View 1 Replies
View Related
Sep 23, 2013
I want to create a macro button that will open a specific webpage. In a cell. I need to press the CTRL + link to access the site, but within a macro I don't know
View 1 Replies
View Related
Mar 27, 2014
I'm trying to enter some in a website and submit it, I'm using this code:
Code:
Sub LoginToSite2()
Dim IE As Object
Dim IEdoc As Object
Dim Password As Object
[Code]....
However I can't get to work for this website, I'm getting Run-time error "91" Object variable or With block variable not set I can only post the source code of the website (below). Also it contains frames, maybe that's part of the problem.
View 3 Replies
View Related
May 16, 2014
I've been working with this script to import and paste a webpage to a worksheet. Once the webpage is on a worksheet I can pull it into VBA easily enough but I was wondering if there's a way to "paste" the webpage directly into an array in VBA and skip the paste to worksheet step?
Code:
Set IE = CreateObject("InternetExplorer.Application")
With IE
.Navigate theURL
IE.Visible = False
End With
[Code]....
View 4 Replies
View Related
Jun 6, 2009
I am using VBA to pull a web query to get sone info from an intranet site.
Unfortunately, the data it pulls is usually old. I've found the only solution is to open a browser manually and to click the Reload button on the web page then navigate back to my workbook and run it again. Then it will pull the most up to date info.
I've tried several ways of forcing a refresh of the data, but have not figured it out yet.
I have been trying the past couple of hours to use the send keys button to tab down to the button on the page I need and to hit enter, thus forcing a refresh, but I can't seem to get that to work either.
View 9 Replies
View Related
Jun 6, 2009
If a1=True i want to navigate to a webpage how can this be done?
View 9 Replies
View Related
Nov 6, 2009
I'm trying to click a link on a webpage I already have open.
For example say I have google.com open and I wanted to click 'Images' at the top of the page without using the code...
ie.navigate("http://images.google.com/imghp?hl=en&tab=wi")
(I can't use this code b/c the page i'm trying to access requires I click the link)
Is there a way to identify the link on the page by its name and then "click" it?
I've been trying to find code for this all over the place, and it seems impossible to find it
View 9 Replies
View Related
Oct 14, 2009
I'm trying to use the CreateObject("MSXML2.XMLHTTP") technique to log in to my Amazon Seller Central account and extract the HTML page source of the URL. However, I simply haven't been successful with getting it to log in. I should note that I know very little about this technique, and simply copied and slightly modified the code from a website. It works when there is no log in. Here's an example of the
Dim URL As String: Dim HttpRequest As Object
URL = "https://sellercentral.amazon.com/gp/orders/fba-order-details.html?ie=UTF8&orderID=123-4940-3939039"
Set HttpRequest = CreateObject("MSXML2.XMLHTTP")
HttpRequest.Open "Get", URL, False
HttpRequest.Send
This is the code I use to get the page source when I don't need to log in. However, the URL I'm now trying to get to requires me to log in first. I've come across a couple different sites that offer the following techniques:
1) URL = "https://usernameassword@sellercentral.amazon.com............."
2) HttpRequest.Open "Post", URL, False, username, password
View 2 Replies
View Related
Sep 3, 2007
I am using excel vba to control IE page with a variety of forms / inputs .
Set ie = CreateObject("InternetExplorer.application")
ie.Visible = True
ie.Navigate (URL)
Call ie_wait(ie)
ie.Document.all.loginID.Value = USERNAME
ie.Document.all.PASSWORD.Value = PASSWORD
ie.Document.forms(1).submit 'login
I am trying to control a drop down box in a HTML page
<select name="attv_39721_5004" onchange="executeRules(this, '39721', '5004', 'attv_', 'Choose...')">
<option value="0">Choose...</option>
<option value="8140">1</option>
<option value="8141">2</option>
<option value="8142" selected="selected" >3</option>
</select>
and i will normally use
ie.Document.all.attv_39721_5004.SelectedIndex = newS
The problem is the unique name for the html select drop-down box changes always!! i.e. it may not always be "attv_39721_5004"
Thus I am not able to use the above method. Does anyone have a different method to changing this values?
View 3 Replies
View Related
