Skip to content

Commit 431f9f5

Browse files
author
Aaron Gonzales
committed
updated docs for counts endpoint
1 parent 28bce75 commit 431f9f5

3 files changed

Lines changed: 327 additions & 32 deletions

File tree

‎README.rst‎

Lines changed: 96 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -122,8 +122,8 @@ example::
122122
--no-print-stream
123123

124124

125-
Using the Twitter Search API Within a Python Program
126-
====================================================
125+
Using the Twitter Search API within Python
126+
==========================================
127127

128128
Working with the API within a Python program is straightforward both for
129129
Premium and Enterprise clients.
@@ -266,6 +266,10 @@ Let's see how it goes:
266266
using bearer token for authentication
267267
268268
269+
By default, tweet payloads are lazily parsed into a ``Tweet`` object. An
270+
overwhelming number of tweet attributes are made available directly, as
271+
such:
272+
269273
.. code:: python
270274
271275
[(tweet.id, tweet.all_text, tweet.hashtags) for tweet in tweets[0:10]]
@@ -351,10 +355,96 @@ easily extractable.
351355
352356
353357
354-
Let's make a new rule and pass it dates this time. ``gen_rule_payload``
355-
takes dates of the forms ``YYYY-mm-DD`` and ``YYYYmmDD``. Note that this
356-
will only work with the full archive search option, which is available
357-
to my account only via the enterprise options.
358+
Counts API
359+
----------
360+
361+
We can also use the counts api to get counts of tweets that match our
362+
rule. Each request will return up to *30* results, and each count
363+
request can be done on a minutely, hourly, or daily basis. There is a
364+
utility function that will convert your regular endpoint to the count
365+
endpoint.
366+
367+
The process is very similar to grabbing tweets, but has some minor
368+
differneces.
369+
370+
**Caveat - premium sandbox environments do NOT have access to the counts
371+
API.**
372+
373+
.. code:: python
374+
375+
from twittersearch import change_to_count_endpoint
376+
count_endpoint = change_to_count_endpoint("https://gnip-api.twitter.com/search/fullarchive/accounts/shendrickson/ogformat.json")
377+
378+
count_args = {"username": "agonzales@twitter.com",
379+
"password": os.environ["TWITTER_SEARCH_PW"],
380+
"endpoint": count_endpoint,
381+
}
382+
383+
count_rule = gen_rule_payload("beyonce", count_bucket="day")
384+
385+
counts = collect_results(count_rule, result_stream_args=count_args)
386+
387+
388+
.. parsed-literal::
389+
390+
using username and password for authentication
391+
392+
393+
Our results are pretty straightforward and can be rapidly used.
394+
395+
.. code:: python
396+
397+
counts
398+
399+
400+
.. parsed-literal::
401+
402+
[{'count': 135320, 'timePeriod': '201711100000'},
403+
{'count': 68532, 'timePeriod': '201711090000'},
404+
{'count': 67138, 'timePeriod': '201711080000'},
405+
{'count': 73017, 'timePeriod': '201711070000'},
406+
{'count': 52290, 'timePeriod': '201711060000'},
407+
{'count': 79338, 'timePeriod': '201711050000'},
408+
{'count': 200519, 'timePeriod': '201711040000'},
409+
{'count': 160512, 'timePeriod': '201711030000'},
410+
{'count': 220683, 'timePeriod': '201711020000'},
411+
{'count': 190959, 'timePeriod': '201711010000'},
412+
{'count': 121580, 'timePeriod': '201710310000'},
413+
{'count': 39473, 'timePeriod': '201710300000'},
414+
{'count': 35441, 'timePeriod': '201710290000'},
415+
{'count': 36198, 'timePeriod': '201710280000'},
416+
{'count': 36149, 'timePeriod': '201710270000'},
417+
{'count': 34197, 'timePeriod': '201710260000'},
418+
{'count': 41497, 'timePeriod': '201710250000'},
419+
{'count': 47648, 'timePeriod': '201710240000'},
420+
{'count': 49087, 'timePeriod': '201710230000'},
421+
{'count': 44945, 'timePeriod': '201710220000'},
422+
{'count': 54865, 'timePeriod': '201710210000'},
423+
{'count': 74324, 'timePeriod': '201710200000'},
424+
{'count': 76643, 'timePeriod': '201710190000'},
425+
{'count': 115587, 'timePeriod': '201710180000'},
426+
{'count': 82581, 'timePeriod': '201710170000'},
427+
{'count': 72372, 'timePeriod': '201710160000'},
428+
{'count': 64522, 'timePeriod': '201710150000'},
429+
{'count': 56092, 'timePeriod': '201710140000'},
430+
{'count': 80265, 'timePeriod': '201710130000'},
431+
{'count': 137717, 'timePeriod': '201710120000'},
432+
{'count': 86203, 'timePeriod': '201710110000'}]
433+
434+
435+
436+
Dated searches / Full Archive Search
437+
------------------------------------
438+
439+
Let's make a new rule and pass it dates this time.
440+
441+
``gen_rule_payload`` takes dates of the forms ``YYYY-mm-DD`` and
442+
``YYYYmmDD``.
443+
444+
**Note that this will only work with the full archive search option**,
445+
which is available to my account only via the enterprise options. Full
446+
archive search will likely require a different endpoint or access
447+
method; please see your developer console for details.
358448

359449
.. code:: python
360450
@@ -380,9 +470,6 @@ to my account only via the enterprise options.
380470
.. code:: python
381471
382472
[(str(tweet.created_at_datetime), tweet.all_text, tweet.hashtags) for tweet in tweets[0:10]]
383-
384-
385-
386473
387474
388475
.. parsed-literal::
@@ -409,5 +496,3 @@ to my account only via the enterprise options.
409496
('2017-10-25 20:15:19',
410497
'Setting up at @CampFlogGnaw https://t.co/nVq8QjkKsf',
411498
[])]
412-
413-

‎examples/api_example.ipynb‎

Lines changed: 121 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,9 @@
2828
{
2929
"cell_type": "code",
3030
"execution_count": 1,
31-
"metadata": {},
31+
"metadata": {
32+
"collapsed": true
33+
},
3234
"outputs": [],
3335
"source": [
3436
"import os\n",
@@ -48,7 +50,9 @@
4850
{
4951
"cell_type": "code",
5052
"execution_count": 22,
51-
"metadata": {},
53+
"metadata": {
54+
"collapsed": true
55+
},
5256
"outputs": [],
5357
"source": [
5458
"# set your environment variables here for enterprise access if you need to\n",
@@ -180,6 +184,13 @@
180184
"tweets = collect_results(rule, max_results=500, result_stream_args=premium_search_args) # change this if you need to"
181185
]
182186
},
187+
{
188+
"cell_type": "markdown",
189+
"metadata": {},
190+
"source": [
191+
"By default, tweet payloads are lazily parsed into a `Tweet` object. An overwhelming number of tweet attributes are made available directly, as such:"
192+
]
193+
},
183194
{
184195
"cell_type": "code",
185196
"execution_count": 7,
@@ -311,7 +322,114 @@
311322
"cell_type": "markdown",
312323
"metadata": {},
313324
"source": [
314-
"Let's make a new rule and pass it dates this time. `gen_rule_payload` takes dates of the forms `YYYY-mm-DD` and `YYYYmmDD`. Note that this will only work with the full archive search option, which is available to my account only via the enterprise options."
325+
"## Counts API\n",
326+
"\n",
327+
"We can also use the counts api to get counts of tweets that match our rule. Each request will return up to *30* results, and each count request can be done on a minutely, hourly, or daily basis. There is a utility function that will convert your regular endpoint to the count endpoint.\n",
328+
"\n",
329+
"The process is very similar to grabbing tweets, but has some minor differneces.\n",
330+
"\n",
331+
"\n",
332+
"**Caveat - premium sandbox environments do NOT have access to the counts API.**\n",
333+
"\n"
334+
]
335+
},
336+
{
337+
"cell_type": "code",
338+
"execution_count": 23,
339+
"metadata": {},
340+
"outputs": [
341+
{
342+
"name": "stderr",
343+
"output_type": "stream",
344+
"text": [
345+
"using username and password for authentication\n"
346+
]
347+
}
348+
],
349+
"source": [
350+
"from twittersearch import change_to_count_endpoint\n",
351+
"count_endpoint = change_to_count_endpoint(\"https://gnip-api.twitter.com/search/fullarchive/accounts/shendrickson/ogformat.json\")\n",
352+
"\n",
353+
"count_args = {\"username\": \"agonzales@twitter.com\",\n",
354+
" \"password\": os.environ[\"TWITTER_SEARCH_PW\"],\n",
355+
" \"endpoint\": count_endpoint,\n",
356+
" }\n",
357+
"\n",
358+
"count_rule = gen_rule_payload(\"beyonce\", count_bucket=\"day\")\n",
359+
"\n",
360+
"counts = collect_results(count_rule, result_stream_args=count_args)"
361+
]
362+
},
363+
{
364+
"cell_type": "markdown",
365+
"metadata": {},
366+
"source": [
367+
"Our results are pretty straightforward and can be rapidly used."
368+
]
369+
},
370+
{
371+
"cell_type": "code",
372+
"execution_count": 29,
373+
"metadata": {},
374+
"outputs": [
375+
{
376+
"data": {
377+
"text/plain": [
378+
"[{'count': 135320, 'timePeriod': '201711100000'},\n",
379+
" {'count': 68532, 'timePeriod': '201711090000'},\n",
380+
" {'count': 67138, 'timePeriod': '201711080000'},\n",
381+
" {'count': 73017, 'timePeriod': '201711070000'},\n",
382+
" {'count': 52290, 'timePeriod': '201711060000'},\n",
383+
" {'count': 79338, 'timePeriod': '201711050000'},\n",
384+
" {'count': 200519, 'timePeriod': '201711040000'},\n",
385+
" {'count': 160512, 'timePeriod': '201711030000'},\n",
386+
" {'count': 220683, 'timePeriod': '201711020000'},\n",
387+
" {'count': 190959, 'timePeriod': '201711010000'},\n",
388+
" {'count': 121580, 'timePeriod': '201710310000'},\n",
389+
" {'count': 39473, 'timePeriod': '201710300000'},\n",
390+
" {'count': 35441, 'timePeriod': '201710290000'},\n",
391+
" {'count': 36198, 'timePeriod': '201710280000'},\n",
392+
" {'count': 36149, 'timePeriod': '201710270000'},\n",
393+
" {'count': 34197, 'timePeriod': '201710260000'},\n",
394+
" {'count': 41497, 'timePeriod': '201710250000'},\n",
395+
" {'count': 47648, 'timePeriod': '201710240000'},\n",
396+
" {'count': 49087, 'timePeriod': '201710230000'},\n",
397+
" {'count': 44945, 'timePeriod': '201710220000'},\n",
398+
" {'count': 54865, 'timePeriod': '201710210000'},\n",
399+
" {'count': 74324, 'timePeriod': '201710200000'},\n",
400+
" {'count': 76643, 'timePeriod': '201710190000'},\n",
401+
" {'count': 115587, 'timePeriod': '201710180000'},\n",
402+
" {'count': 82581, 'timePeriod': '201710170000'},\n",
403+
" {'count': 72372, 'timePeriod': '201710160000'},\n",
404+
" {'count': 64522, 'timePeriod': '201710150000'},\n",
405+
" {'count': 56092, 'timePeriod': '201710140000'},\n",
406+
" {'count': 80265, 'timePeriod': '201710130000'},\n",
407+
" {'count': 137717, 'timePeriod': '201710120000'},\n",
408+
" {'count': 86203, 'timePeriod': '201710110000'}]"
409+
]
410+
},
411+
"execution_count": 29,
412+
"metadata": {},
413+
"output_type": "execute_result"
414+
}
415+
],
416+
"source": [
417+
"counts"
418+
]
419+
},
420+
{
421+
"cell_type": "markdown",
422+
"metadata": {},
423+
"source": [
424+
"## Dated searches / Full Archive Search\n",
425+
"\n",
426+
"\n",
427+
"Let's make a new rule and pass it dates this time.\n",
428+
"\n",
429+
"`gen_rule_payload` takes dates of the forms `YYYY-mm-DD` and `YYYYmmDD`.\n",
430+
"\n",
431+
"\n",
432+
"**Note that this will only work with the full archive search option**, which is available to my account only via the enterprise options. Full archive search will likely require a different endpoint or access method; please see your developer console for details."
315433
]
316434
},
317435
{

0 commit comments

Comments
 (0)