Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Making The Most of Your Data Store

Making The Most of Your Data Store

Making The Most of Your Data Store
Or (hopefully) some tricks you probably didn’t know.

Dougal Matthews

August 01, 2012
Tweet

More Decks by Dougal Matthews

Other Decks in Technology

Transcript

  1. Making The Most
    of Your Data Store
    Or (hopefully) some tricks you probably didn’t know.
    Dougal Matthews
    dougalmatthews.com
    twitter.com/d0ugal
    Wednesday, 1 August 12

    View Slide

  2. Wednesday, 1 August 12

    View Slide

  3. Wednesday, 1 August 12

    View Slide

  4. Redis
    Wednesday, 1 August 12

    View Slide

  5. >>> from redis import Redis
    >>> r = Redis()
    >>> pubsub = r.pubsub()
    >>> pubsub.subscribe("queue_name")
    >>> r.publish("queue_name", "My Message.")
    1L
    >>> print pubsub.listen().next()
    {'pattern': None, 'type': 'subscribe',
    'channel': 'queue_name', 'data': 1L}
    >>> print pubsub.listen().next()
    {'pattern': None, 'type': 'message',
    'channel': 'queue_name', 'data': 'My
    Message.'}
    Pub Sub
    Wednesday, 1 August 12

    View Slide

  6. Config
    slowlog-log-slower-than 10000
    slowlog-max-len 128
    Usage
    SLOWLOG LEN
    SLOWLOG GET 10
    SLOWLOG RESET
    slowlog
    Wednesday, 1 August 12

    View Slide

  7. >>> from redis import Redis
    >>> r = Redis()
    >>> r.set('foo', 'bar')
    >>> pipe = r.pipeline()
    >>> # Queue up operations
    >>> pipe.get('foo')
    >>> pipe.set('foo', 'baz')
    >>> pipe.get('foo')
    >>> pipe.execute()
    ['bar', True, 'baz']
    redis-py Pipelines
    Wednesday, 1 August 12

    View Slide

  8. - django-redis-cache
    - django-redis-sessions
    - Celery Backend
    - thoonk.py
    Libs
    Wednesday, 1 August 12

    View Slide

  9. PostgreSQL
    Wednesday, 1 August 12

    View Slide

  10. CREATE TABLE example (
    id serial PRIMARY KEY,
    data hstore);
    INSERT INTO example (data) VALUES
    ('name => "John Smith", age => 28, gender => "M"'),
    ('name => "Jane Smith", age => 24');
    SELECT id, data->'name' FROM example;
    SELECT id, data->'age' FROM example
    WHERE data->'age' >= '25';
    hstore
    Wednesday, 1 August 12

    View Slide

  11. CREATE LANGUAGE plpythonu;
    CREATE FUNCTION pymax (a integer, b integer)
    RETURNS integer
    AS $$
    if a > b:
    return a
    return b
    $$ LANGUAGE plpythonu;
    SELECT pymax(1, 2);
    PL/Python
    Wednesday, 1 August 12

    View Slide

  12. import psycopg2
    from psycopg2.extras import wait_select
    aconn = psycopg2.connect(..., async=1)
    wait_select(aconn)
    acurs = aconn.cursor()
    acurs.execute("SELECT pg_sleep(5);
    SELECT * FROM example;")
    wait_select(acurs.connection)
    acurs.fetchone()[0]
    Psycopg2 and Async
    Wednesday, 1 August 12

    View Slide

  13. CREATE TABLE detailed_example(
    more_data hstore
    ) INHERITS (example);
    Table Inheritance
    Wednesday, 1 August 12

    View Slide

  14. - django-hstore
    Libs
    Wednesday, 1 August 12

    View Slide

  15. MySQL
    Wednesday, 1 August 12

    View Slide

  16. Just Kidding.
    Use Postgres.
    Wednesday, 1 August 12

    View Slide

  17. Tekpub: Why Not MySQL?
    http://s.do.ugal.me/IQ0z
    Rob Conery - Five Things You Didn't
    Know About PostGresSQL
    http://s.do.ugal.me/IQvc
    ...but more seriously
    Wednesday, 1 August 12

    View Slide

  18. Questions?
    Dougal Matthews
    dougalmatthews.com
    twitter.com/d0ugal
    Image Credits;
    http://www.flickr.com/photos/zooboing/4182512221/
    http://www.flickr.com/photos/r80o/5549288/
    http://www.flickr.com/photos/gavinmroy/4638958958/
    (Or your favourite DB feature?)
    Slides: http://s.do.ugal.me/IRgR
    Wednesday, 1 August 12

    View Slide

  19. We are hiring Python developers.
    www.artirix.com/about-us/jobs/
    Wednesday, 1 August 12

    View Slide