Feed Me! How To Get Your External Data Into Drupal

Session from DrupalCamp Scotland 2014

It's quite common to require data to be transferred from an external source into Drupal. Maybe you have 100 staff records sitting in a spreadsheet that you'd rather not have to manually enter. Or maybe you need a more sophisticated setup that can import records from a 3rd party system at regular intervals. Feeds may provide the ideal solution for importing your data. This session will introduce the Feeds module from the beginning and will explain the anatomy of Feeds - importers, fetchers, parsers, processors and mappers. We will also walk through the process of setting up an import from a CSV file. Feeds can be customised using hooks and custom plugins and this session will also provide some useful info for developers by looking at how the Feeds module can be extended.​

Transcript

1. DrupalCamp Scotland 2014 How to get your external data into Drupal John Hume Drupal.org: @littledynamo Twitter: @littledynamo1
2. What is the Feeds module? “Import or aggregate data as nodes, users, taxonomy terms or simple database records.” drupal.org/project/feeds Basically … Feeds allows you to import external data into Drupal
3. A bit of history … Released in 2009 by DevelopmentSeed Part of Managing News distribution Written by Frank Febbraro @febbraro Maintained by Chris Leppanen @twistor Successor of Feed API and Feed Mapper Currently in Alpha – but don’t let that put you off!
4. FeedsS
5. Case Study – Career Timeline
6. Case Study – Career Timeline • Timeline event content type • Over 600 nodes to be created • Data held in multiple spreadsheets Feeds to the Rescue 
7. The Fetcher
8. The Parser
9. The Processor
10. Feeds Importer Fetcher Parser Processor Mapping Feeds Anatomy Basic Settings
11. Fetchers Out of the Box Contributed File Upload HTTP PubSubHubbub Callback Directory Oauth HTTP Advanced Authentication Clients SOAP OAI-PMH Mailhandler LDAP SQL Web Services Client HTTP Append Headers FileMaker
12. Parsers Out of the Box Contributed RSS / Atom CSV OPML XPath iCal Flickr YouTube Vimeo LDAP FileMaker Slideshare Clients Regex Wordpress SharePoint LOTS MORE! XML Sitemap Simplepie
13. Processors Out of the Box Contributed Node User Taxonomy term Feed Node Data Comment Commerce product RedHen Node (Unique) Self node User contacts Node Multisource
14. CSV Import Demo 1. Create content type 2. Create CSV file 3. Add basic settings 4. Setup Fetcher, Parser and Processor 5. Add mapping 6. Import
15. Feeds Tamper drupal.org/project/feeds_tamper Provides a UI for modifying data before it gets saved Examples: • Replace every instance of 'dog' with 'cat'. • Combine separate 'firstname' and 'lastname' fields into one 'name' field. • Break a comma separated list of words into Taxonomy terms or a multivalued text field.
16. Source : http://ygerasimov.com/feeds-cheatsheet-for-developers Customising Feeds – Program Flow
17. Customising Feeds - Hooks Source: http://drupalcontrib.org/api/drupal/contributions!feeds!feeds.api.php/7
18. Source : http://ygerasimov.com/feeds-cheatsheet-for-developers Before Import
19. Source : http://ygerasimov.com/feeds-cheatsheet-for-developers After Parse
20. Source : http://ygerasimov.com/feeds-cheatsheet-for-developers Presave
21. Source : http://ygerasimov.com/feeds-cheatsheet-for-developers After Import
22. Feeds Hook Demo hook_feeds_after_parse()
23. Custom Plugins! Create your own Fetcher / Parser / Processor • Declare and Implement a Ctools plugin • Extend FeedsFetcher, FeedsParser, FeedsProcessor • If similar plugin exists, extend that class instead
24. Case Study – Staff Import from Staff Database stored in FileMaker Old D6 Custom Module Increasing requirement for FileMaker => Drupal Custom Fetcher and Parser
25. Case Study – Staff Import from feeds_filemaker_feeds_plugins() configForm(), sourceForm() - Settings getRaw()– retrieve the data from FileMaker parse()- normalise the data FeedsNodeProcessor does it’s thing
26. Resources • List of Feeds contributed Fetchers, Parsers, Processors and Mappers - https://drupal.org/node/856644 • Feeds cheat sheet for developers - http://ygerasimov.com/feeds-cheatsheet-for- developers • Developers guide to feeds - https://drupal.org/node/622700
27. Image Credits • Speedy Boy Delivery - https://www.flickr.com/photos/42424413@N06/5 954193181/ • Lego Chef - https://www.flickr.com/photos/dunechaser/16150 9118 • Storm Troopers - https://www.flickr.com/photos/st3f4n/444814037 7