R
Rick
RE: parsing up cookies / MSXML.XMLHTTP4.0 / setRequestHeader "Cookie" /
getAllResponseHeaders / HTTP / MSXML.DLL
Everyone must run into this problem - logging in to a web site,
authenticating, getting the session information from the server in assorted
session and persistent cookies and then subsequently sending all the proper
cookies on with subsequent HTTP requests.
I'm NOT opting for WINHTTP5.DLL (Microsoft WinHTTP Services) because I want
to be sure a wide range of windows users can use the code. Trying NOT to use
shdocvw.dll (Internet Explorer) either because it can be very slow with all
the JavaScript, pictures and pixel rendering. All I'm interested in on
subsequent "GET"s are the raw HTTP response text streams.
Sample code below, (Through fetching and parsing the cookies) -- very rough,
not robust.
Couple of questions:
1. Do I ever need to worry about "Set-Cookie2:" ?
2. When I make a subsequent HTTP "GET" request, for instance to
"http://finance.yahoo.com," do I need to include all cookies for BOTH:
.finance.yahoo.com AND
.yahoo.com ?
3. Are cookies always delimited with "; " or sometimes no white space ";"
4. Anyone have any better ideas for how to parse up the "Set-Cookie: "
lines? I feel like an amateur here. Every browser does it with speed and
grace - I'm a neophyte in parsing 101, especially with optional and random
placed arguments.
5. Any tips on error trapping MSXML.DLL, timing it out, etc., so Excel won't
hang if the Internet or the web site go down.
6. Any tips on spitting the cookies back at the server using MSXML.DLL
SetRequestHeader? Can multiple cookies be set at one time. Any "got-yas"?
All very much appreciated!
Rick
(e-mail address removed)
ITSdoc stands for Investment and Trading Systems Documentation - the free
encyclopedia of documentation for coders. http://ITSdoc.org
Sub test()
Dim sHeader As String
sHeader = fetchHeader("http://finance.yahoo.com")
Debug.Print parseCookies(sHeader)
End Sub
Function fetchHeader(sURL As String) As String
' returns an HTTP response header after going to sURL
' need a reference to MSXML.DLL
Dim xmlhttp As Object
Set xmlhttp = CreateObject("MSXML2.Serverxmlhttp.4.0")
With xmlhttp
.Open "GET", sURL, False
.send
While .readyState <> 4: .waitForResponse 1000: Wend
fetchHeader = .getAllResponseHeaders()
End With
End Function
Function parseCookies(sHeaderString As String) As String
'You give it a HTTP header string and it chops out all the cookies
Dim i As Integer
Dim j As Integer
Dim aHeaderLine() As String
Dim aCookies() As String
Dim aCookieParts() As String
Dim cookieTray As String
aHeaderLine() = Split(sHeaderString, Chr(13) & Chr(10))
For i = 0 To UBound(aHeaderLine())
If Left(aHeaderLine(i), 12) = "Set-Cookie: " Then 'we have a cookie
' anyone find any "Set-Cookie2: " out there???
aCookieParts() = Split(Mid(aHeaderLine(i), 13), "; ") 'is the whitespace
always there?
cookieTray = cookieTray & aCookieParts(0) & vbTab
For j = 0 To UBound(aCookieParts())
If Left(aCookieParts(j), 7) = "domain=" Then 'got the domain
cookieTray = cookieTray & Mid(aCookieParts(j), 8) & vbCrLf
End If
Next
End If
Next
parseCookies = cookieTray
End Function
getAllResponseHeaders / HTTP / MSXML.DLL
Everyone must run into this problem - logging in to a web site,
authenticating, getting the session information from the server in assorted
session and persistent cookies and then subsequently sending all the proper
cookies on with subsequent HTTP requests.
I'm NOT opting for WINHTTP5.DLL (Microsoft WinHTTP Services) because I want
to be sure a wide range of windows users can use the code. Trying NOT to use
shdocvw.dll (Internet Explorer) either because it can be very slow with all
the JavaScript, pictures and pixel rendering. All I'm interested in on
subsequent "GET"s are the raw HTTP response text streams.
Sample code below, (Through fetching and parsing the cookies) -- very rough,
not robust.
Couple of questions:
1. Do I ever need to worry about "Set-Cookie2:" ?
2. When I make a subsequent HTTP "GET" request, for instance to
"http://finance.yahoo.com," do I need to include all cookies for BOTH:
.finance.yahoo.com AND
.yahoo.com ?
3. Are cookies always delimited with "; " or sometimes no white space ";"
4. Anyone have any better ideas for how to parse up the "Set-Cookie: "
lines? I feel like an amateur here. Every browser does it with speed and
grace - I'm a neophyte in parsing 101, especially with optional and random
placed arguments.
5. Any tips on error trapping MSXML.DLL, timing it out, etc., so Excel won't
hang if the Internet or the web site go down.
6. Any tips on spitting the cookies back at the server using MSXML.DLL
SetRequestHeader? Can multiple cookies be set at one time. Any "got-yas"?
All very much appreciated!
Rick
(e-mail address removed)
ITSdoc stands for Investment and Trading Systems Documentation - the free
encyclopedia of documentation for coders. http://ITSdoc.org
Sub test()
Dim sHeader As String
sHeader = fetchHeader("http://finance.yahoo.com")
Debug.Print parseCookies(sHeader)
End Sub
Function fetchHeader(sURL As String) As String
' returns an HTTP response header after going to sURL
' need a reference to MSXML.DLL
Dim xmlhttp As Object
Set xmlhttp = CreateObject("MSXML2.Serverxmlhttp.4.0")
With xmlhttp
.Open "GET", sURL, False
.send
While .readyState <> 4: .waitForResponse 1000: Wend
fetchHeader = .getAllResponseHeaders()
End With
End Function
Function parseCookies(sHeaderString As String) As String
'You give it a HTTP header string and it chops out all the cookies
Dim i As Integer
Dim j As Integer
Dim aHeaderLine() As String
Dim aCookies() As String
Dim aCookieParts() As String
Dim cookieTray As String
aHeaderLine() = Split(sHeaderString, Chr(13) & Chr(10))
For i = 0 To UBound(aHeaderLine())
If Left(aHeaderLine(i), 12) = "Set-Cookie: " Then 'we have a cookie
' anyone find any "Set-Cookie2: " out there???
aCookieParts() = Split(Mid(aHeaderLine(i), 13), "; ") 'is the whitespace
always there?
cookieTray = cookieTray & aCookieParts(0) & vbTab
For j = 0 To UBound(aCookieParts())
If Left(aCookieParts(j), 7) = "domain=" Then 'got the domain
cookieTray = cookieTray & Mid(aCookieParts(j), 8) & vbCrLf
End If
Next
End If
Next
parseCookies = cookieTray
End Function