[HELP] HTTP requests with requests module [SOLVED]

[Tizzy]

Hi everyone! It's been a while since I posted....been busy and ran into some setbacks (macbook stolen, private api endpoint for my project changed amongst others ... :( ).

Anyhow, I'm having some issues using the requests module and was hoping somebody here might have some experience/insight to divulge.

The problem:
I'm able to successfully access a particular https endpoint using a post request using multiple REST api testing apps (Rested for OS X, and RESTer for iOS) - For whatever reason I'm getting an entirely different response when I try to do the same thing using the requests module in python.

The requests that I make from the two apps successfully [code 200] use no parameters, and an http Body is attached along with them that contains the value from the dict dataToPOST shown below exactly.

Python's requests module gives me a [code 500]

I used the slightly more complicated request/prepare and session objects from the requests module in order to be able to see the headers etc of the request object in the state it is sent. (rather than requests.post(...))

One interesting thing to note: requests module seems to add a "Content-Length: 392" header that I can't get rid of. I tried setting it manually to Null/none but after the request was prepared it was overridden back to the 392 value.
-I also tested adding this content-length 392 value to both of the REST apps making the request and it didn't seem to affect anything there so I don't think it's the issue.

I'm guessing the issue has to do with the dataToPOST dict I insert into my request's "data=" parameter, but I'm at a loss to understand what it is.
Again, the same dict, literally copy-pasted into the http body forms of the testing apps works flawlessly.

I've tried changing my python file's encoding to both utf-8 and ascii and it doesn't seem to change the result. I chose the headers to mirror exactly those used by the testing apps, but no cigar.

Help?

dataToPOST = {
	
	
		"token": "[censored",
	
		"messageType": "[censored]",
		"apiUrl": "\/[censored]\/[censored]",
		"longitude": [censored],
		"altitude": [censored],
		"deviceCarrier": "T-Mobile",
		"horizontalAccuracy": 10,
		"apiParameters": {
			"epoch": 145327447925,
		
			"_LOCALE_": "en",
			"token": "[censored]"
		},
		"language": "en",
	
		"apiMethod": "GET",
		"deviceSignalStrength": 11,
		"speed": 0,
		"deviceIds": {
			"advertiserId": "[censored]",
			"uberId": "[censored]",
			"bluetoothMac": "[censored]"
		}
		}

partnerInfoHeaders = {"Connection":"keep-alive","Content-Length":"Null","Accept":"*/*","Content-Type":"text/plain","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us"}

partnerInfoRequestObject = requests.Request("POST","https://[censored].com/", data = dataToPOST,headers = partnerInfoHeaders)

partnerInfoRequestPrepared=partnerInfoRequestObject.prepare()

print "prepared request headers: ",partnerInfoRequestPrepared.headers

requestSesh = requests.Session()
partnerInfoResponse = requestSesh.send(partnerInfoRequestPrepared)

[JonB]

Have you tried recording the header on the server side? Or wireshark?

[Tizzy]

[UPDATE]
So I messed with Charles proxy ... have some weird behavior to report.

I'm able to proxy the testing apps (OS X and iOS) using the api endpoint,
but I'm not able to proxy the requests module making requests with some caveats:

OSX:
-requests.Session object: 500 response, doesn't show up in my sniffing proxy at all
-requests.Request object throws an ssl certificate error (WITHOUT TRYING TO SSL PROXY IT DOES THE SAME 500 ERROR)
-RESTED http testing app: 200 response + shows up in proxy

iOS:
-requests.Session object: gives same 500 response, doesn't show up in proxy
-requests.Request object gives same 500 response, doesn't show up in the proxy
-RESTer app : 200 response + shows up in proxy

All this leads me to believe the request module is doing some weird low-level networking shiz that bypasses system-wide proxy settings.... What do you guys think?

Is there a requests alternative anyone would suggest trying?

....
I don't have access to the server side...unless you mean inspecting the response headers in which case yes, they seem to be the same for the two different testing apps.... I'm trying to censor what I post since this is a private api. I've never used wireshark but I've used charles proxy, I can give it a go.

[Tizzy]

[SOLVED]

So the solution was to change the "data = " parameter to "json = " in the request.

(ALSO FOR THE xCODE TEMPLATE HAD TO UPDATE THE VERSION OF REQUESTS. 2.9.1 seems to be safely working)

Not working (500 error):

partnerInfoRequestObject = requests.Request("POST","https://[censored].com/", data = dataToPOST,headers = partnerInfoHeaders)

Working:

partnerInfoRequestObject = requests.Request("POST","https://[censored].com/", json = dataToPOST,headers = partnerInfoHeaders)

As a sidenote, what also half-worked was wrapping my dict in triple quotes for the "data=" parameter - that would give me a 200 response but a different internal server error message. Just changing it to "json=" seemed to do the trick. I was not aware you could just plug a dict in where json was expected.