mysql - Big Query with Python -

-

is there anyway run queries repeatedly on google big query using python script?

i want query dataset using google big query platform weeks data , want on year. bit tedious query dataset 52 times. instead prefer write python script(as know python).

i hope point me in right direction regarding this.

bigquery supplies client libraries several languages -- see https://cloud.google.com/bigquery/client-libraries -- , in particular python, docs @ https://developers.google.com/resources/api-libraries/documentation/bigquery/v2/python/latest/?_ga=1.176926572.834714677.1415848949 (you'll need follow hyperlinks understand docs).

https://cloud.google.com/bigquery/bigquery-api-quickstart gives example of command-line program, in either java or python, uses google bigquery api run query on 1 of available sample datasets , display result. after imports, , setting few constants, python script boils down to

  storage = storage('bigquery_credentials.dat')   credentials = storage.get()    if credentials none or credentials.invalid:       # run oauth2 flow default arguments.       credentials = tools.run_flow(flow, storage, tools.argparser.parse_args([]))    http = httplib2.http()   http = credentials.authorize(http)    bigquery_service = build('bigquery', 'v2', http=http)    try:     query_request = bigquery_service.jobs()     query_data = {'query':'select top( title, 10) title, count(*) revision_count [publicdata:samples.wikipedia] wp_namespace = 0;'}      query_response = query_request.query(projectid=project_number,                                          body=query_data).execute()     print 'query results:'     row in query_response['rows']:       result_row = []       field in row['f']:         result_row.append(field['v'])       print ('\t').join(result_row)    except httperror err:     print 'error:', pprint.pprint(err.content)    except accesstokenrefresherror:     print ("credentials have been revoked or expired, please re-run"            "the application re-authorize")

as see, 30 lines, concerned getting , checking authorization , handling errors. "core" part, net of such considerations, half lines:

    bigquery_service = build('bigquery', 'v2', http=http)     query_request = bigquery_service.jobs()     query_data = {'query':'select top( title, 10) title, count(*) revision_count [publicdata:samples.wikipedia] wp_namespace = 0;'}      query_response = query_request.query(projectid=project_number,                                          body=query_data).execute()     print 'query results:'     row in query_response['rows']:       result_row = []       field in row['f']:         result_row.append(field['v'])       print ('\t').join(result_row)

Comments

Post a Comment

Popular posts from this blog

How to run C# code using mono without Xamarin in Android? -

-
i want create program uses mono runtime run c# code (binary not application). i copied libmono.so unity app , i'm write code using documentation: http://www.mono-project.com/docs/advanced/embedding/ and it's working have problems references can access classes mscorlib.dll , have problems system.reflection.assembly class throwing exception, when print console see: system.missingmethodexception: method not found: 'system.reflection.assembly.op_inequality' can fix it? can use mono runtime in c binary on android? ps. i'm using android tests. edit: found solution loading libraries, need set environment variable mono_path libraries path. but have still problems assembly class. the system.missingmethodexception error have seems .net profile mismatch error, 1 explained here . can use mono runtime in c binary on android? yes, using ndk guess. there alternatives can use instead of rolling own, such xmadevlab .
Read more

c# - SharpSsh Command Execution -

-
Image
i trying create ssh client using c#. using tamir.sharpssh library. having issues sending command , getting appropriate response server. the way have been testing sending on ls command, followed cd.. command, followed ls command again -- see whether or not cd.. command being executed properly. however, there no difference between first ls command , second ls command. not entirely sure doing wrong. perhaps using wrong ssh type. have provided code , output getting. have added output expect. using tamir.sharpssh; namespace sshnetexample { class program { static void main(string[] args) { sshexec ssh = null; try { console.write("-connecting..."); ssh = new sshexec(host, username, password); ssh.connect(); console.writeline("ok ({0}/{1})", ssh.cipher, ssh.mac); console.writeline("server version={0}, client versio...
Read more

python - Specify path of savefig with pylab or matplotlib -

-
i struggling on how find correct way specify save path (or repository) when calling function savefig in matplotlib or pylab. i tried several syntaxes, everytime python console returns : filenotfounderror: [errno 2] no such file or directory: '../mydocs/resource/frames/myimage.png' currently, have written following : pylab.savefig('../mydocs/resource/frames/myimage.png') does know how it? thanks in advance! the tilde operator , variable $home entered strings , therefore not function while saving. have provide either relative path (as did) or provide full path. eg. pylab.savefig("/home/username/desktop/myfig.png") .
Read more