Tweet
Tweet

Slide 1

Slide 1 text

How to walk in the Crowi ʙ How PHPer developed the product of Node.js ʙ PHP Conference 2016 (Tokyo) 2016-11-03 @suzuki

Slide 2

Slide 2 text

Crowi ͷา͖ํ ʙPHPer ͕ Node.js ͷϓϩμΫτΛ৮ΔΑ͏ʹͳΔ·Ͱʙ PHP Conference 2016 (Tokyo) 2016-11-03 @suzuki

Slide 3

Slide 3 text

About me

Slide 4

Slide 4 text

Twitter: @suzuki GitHub: @suzuki

Slide 5

Slide 5 text

PHP Conference Fukuoka 2016 https://speakerdeck.com/suzuki/guzzle-promisewoshi-tuta-fei-tong-qi-chu-li-niyoruapikorufalsegao-su-hua

Slide 6

Slide 6 text

PHP Conference Kansai 2016 https://speakerdeck.com/suzuki/swift-mailer-update

Slide 7

Slide 7 text

Contributor of Crowi https://github.com/crowi/crowi/graphs/contributors

Slide 8

Slide 8 text

What is Crowi?

Slide 9

Slide 9 text

Crowi http://site.crowi.wiki/

Slide 10

Slide 10 text

Features ɾ Wiki ɾ Markdown ɾ Realtime preview ɾ Login by Google Account ɾ Notification to Slack

Slide 11

Slide 11 text

GitHub: crowi/crowi https://github.com/crowi/crowi

Slide 12

Slide 12 text

More information https://medium.com/crowi-book

Slide 13

Slide 13 text

More information https://speakerdeck.com/sotarok/crowi

Slide 14

Slide 14 text

Why am I started to develop the Crowi?

Slide 15

Slide 15 text

When the year-end party in 2015

Slide 16

Slide 16 text

Talk about "esa.io" I like "esa.io". I love their Markdown helper. OK. You create it. I want to use that helper on Crowi. esa logo : https://docs.esa.io/posts/125

Slide 17

Slide 17 text

He is product founder of Crowi https://github.com/crowi/crowi/graphs/contributors

Slide 18

Slide 18 text

I tried to implement

Slide 19

Slide 19 text

Pull request https://github.com/crowi/crowi/pull/38

Slide 20

Slide 20 text

Pull request https://github.com/crowi/crowi/pull/38

Slide 21

Slide 21 text

This is my opportunity
 to develop the Crowi

Slide 22

Slide 22 text

Crowi overview

Slide 23

Slide 23 text

System Diagram .POHP%# &MBTUJDTFBSDI 3FEJT "NB[PO4 4MBDL 3FRVJSF 0QUJPOBM Remote Notification Full text search Data Store Session Data File upload <$SPXJ> /PEFKT (PPHMF Social Login

Slide 24

Slide 24 text

<$SPXJ> $MJFOU4JEF 4FSWFS4JEF Inside Crowi /PEFKT &YQSFTT .POHPPTF 4XJH #SPXTFS K2VFSZ 3FBDU #PPUTUSBQ

Slide 25

Slide 25 text

How to develop?

Slide 26

Slide 26 text

Install and Configuration https://github.com/crowi/crowi/wiki/Install-and-Configuration

Slide 27

Slide 27 text

or

Slide 28

Slide 28 text

GitHub: suzuki/crowi-develop https://github.com/suzuki/crowi-develop

Slide 29

Slide 29 text

What is "crowi-develop" ɾ Setup Docker containers ɾMongoDB ɾRedis ɾElasticsearch ɾ Require ɾNode.js ɾDocker for Mac

Slide 30

Slide 30 text

What is "suzuki/crowi-develop" .POHP%# &MBTUJDTFBSDI 3FEJT "NB[PO4 4MBDL 3FRVJSF 0QUJPOBM <$SPXJ> /PEFKT (PPHMF Prepare these

Slide 31

Slide 31 text

How to initialize
 "crowi-develop"

Slide 32

Slide 32 text

Clone "crowi-develop" repository $ git clone https://github.com/suzuki/crowi-develop

Slide 33

Slide 33 text

Clone "crowi" repository $ cd crowi-develop $ git clone git@github.com:crowi/crowi.git

Slide 34

Slide 34 text

Files in "crowi-develop" LICENSE
 README.md
 crowi/
 data/
 docker-compose.yml Crowi directory Data directory
 for Docker containers

Slide 35

Slide 35 text

Run Docker containers $ cd crowi-develop
 $ docker-compose up -d

Slide 36

Slide 36 text

Run Crowi $ cd crowi-develop/crowi 
 $ PASSWORD_SEED=somesecretstring \
 MONGO_URI=mongodb://localhost/crowi \
 ELASTICSEARCH_URI=localhost:9200 \
 REDIS_URL=localhost \
 FILE_UPLOAD=local \
 node app.js Change your own word

Slide 37

Slide 37 text

Open localhost:3000 http://localhost:3000/

Slide 38

Slide 38 text

Create administrator http://localhost:3000/installer

Slide 39

Slide 39 text

Open administration menu http://localhost:3000/ Administration menu

Slide 40

Slide 40 text

Open search menu http://localhost:3000/admin Search menu

Slide 41

Slide 41 text

Build search index http://localhost:3000/admin/search Build index

Slide 42

Slide 42 text

Daily routine: start up "crowi-develop"

Slide 43

Slide 43 text

Run Docker container $ cd crowi-develop
 $ docker-compose up -d

Slide 44

Slide 44 text

Run Crowi $ cd crowi-develop/crowi 
 $ PASSWORD_SEED=somesecretstring \
 MONGO_URI=mongodb://localhost/crowi \
 ELASTICSEARCH_URI=localhost:9200 \
 REDIS_URL=localhost \
 FILE_UPLOAD=local \
 node app.js Change your own word

Slide 45

Slide 45 text

Daily routine: shutdown "crowi-develop"

Slide 46

Slide 46 text

Shutdown Crowi 
 $ PASSWORD_SEED=somesecretstring \
 MONGO_URI=mongodb://localhost/crowi \
 ELASTICSEARCH_URI=localhost:9200 \
 REDIS_URL=localhost \
 FILE_UPLOAD=local \
 node app.js
 Press Ctrl + C

Slide 47

Slide 47 text

Shutdown Docker container $ docker-compose stop

Slide 48

Slide 48 text

Case study: I want to modify…

Slide 49

Slide 49 text

I want to modify CSS

Slide 50

Slide 50 text

<$SPXJ> $MJFOU4JEF 4FSWFS4JEF CSS /PEFKT &YQSFTT .POHPPTF 4XJH #SPXTFS K2VFSZ 3FBDU #PPUTUSBQ

Slide 51

Slide 51 text

Keywords of using tools ɾ Bootstrap ɾ Sass ɾ Gulp

Slide 52

Slide 52 text

Go to directory $ cd crowi/resource/css/

Slide 53

Slide 53 text

Modify files $ ls -1F
 _admin.scss
 _comment.scss
 _form.scss
 _layout.scss
 _mixins.scss
 _page.scss
 _page_list.scss
 _portal.scss
 _search.scss
 _user.scss
 _utilities.scss
 _variables.scss
 _wiki.scss
 crowi-reveal.scss
 crowi.scss "_NAME.scss" are component "crowi.scss" is main file

Slide 54

Slide 54 text

Build $ ./node_modules/.bin/gulp css

Slide 55

Slide 55 text

Check result ɾ Reload your browser

Slide 56

Slide 56 text

Watch files $ ./node_modules/.bin/gulp watch

Slide 57

Slide 57 text

I want to modify HTML

Slide 58

Slide 58 text

<$SPXJ> $MJFOU4JEF 4FSWFS4JEF HTML (Template) /PEFKT &YQSFTT .POHPPTF 4XJH #SPXTFS K2VFSZ 3FBDU #PPUTUSBQ

Slide 59

Slide 59 text

Keywords of using tools ɾ Swig

Slide 60

Slide 60 text

Swig: template engine https://www.npmjs.com/package/swig

Slide 61

Slide 61 text

Format sample

{{ pagename }}

    {% for author in authors %}
  • {{ author }}
    • {% endfor %}
variable name for loop

Slide 62

Slide 62 text

Swig is similar to the Twig http://twig.sensiolabs.org/

Slide 63

Slide 63 text

Go to directory $ cd crowi/lib/views/

Slide 64

Slide 64 text

Modify files $ ls -1F
 500.html
 _form.html
 admin/
 index.html
 installer.html
 invited.html
 layout/
 login/
 login.html
 mail/
 me/
 modal/
 page.html
 page_list.html
 page_presentation.html
 search.html
 user/
 user_page.html
 widget/ "*.html" are template of Swig

Slide 65

Slide 65 text

Build Nothing to do

Slide 66

Slide 66 text

Check result ɾ Restart Crowi ɾ Reload your browser

Slide 67

Slide 67 text

I want to modify Front-end JavaScript

Slide 68

Slide 68 text

<$SPXJ> $MJFOU4JEF 4FSWFS4JEF Front-end JavaScript /PEFKT &YQSFTT .POHPPTF 4XJH #SPXTFS K2VFSZ 3FBDU #PPUTUSBQ

Slide 69

Slide 69 text

Keywords of using tools ɾ jQuery

Slide 70

Slide 70 text

Go to directory $ cd crowi/resource/js/

Slide 71

Slide 71 text

Modify files $ ls -1F
 app.js
 components/
 crowi-admin.js
 crowi-form.js
 crowi-presentation.js
 crowi.js
 util/

Slide 72

Slide 72 text

Build $ ./node_modules/.bin/gulp [dev]

Slide 73

Slide 73 text

I want to modify Front-end JavaScript (React)

Slide 74

Slide 74 text

<$SPXJ> $MJFOU4JEF 4FSWFS4JEF Front-end JavaScript (React) /PEFKT &YQSFTT .POHPPTF 4XJH #SPXTFS K2VFSZ 3FBDU #PPUTUSBQ

Slide 75

Slide 75 text

Keywords of using tools ɾ Swig ɾ React

Slide 76

Slide 76 text

Our future plan: Use React more

Slide 77

Slide 77 text

Go to directory $ cd crowi/resource/js/

Slide 78

Slide 78 text

Modify files $ ls -1F
 app.js
 components/
 crowi-admin.js
 crowi-form.js
 crowi-presentation.js
 crowi.js
 util/ React components ReactDOM.render()

Slide 79

Slide 79 text

Modify files (components) $ ls -1F components/
 HeaderSearchBox/
 HeaderSearchBox.js
 Page/
 PageList/
 PageListSearch.js
 SearchPage/
 SearchPage.js
 User/

Slide 80

Slide 80 text

Modify Template 4XJH5FNQMBUF EJWJE40.&@*%@'03@3&/%&3EJW React component will render here

Slide 81

Slide 81 text

Modify React renderer (in app.js) const componentMappings = { 'search-top': , 'search-page': , 'page-list-search': , }; Object.keys(componentMappings).forEach((key) => { const elem = document.getElementById(key); if (elem) { ReactDOM.render(componentMappings[key], elem); } }); DOM id (in Swig template)

Slide 82

Slide 82 text

Similar case http://techlife.cookpad.com/entry/2016/10/26/135818

Slide 83

Slide 83 text

Build $ ./node_modules/.bin/gulp [dev]

Slide 84

Slide 84 text

Check result ɾ Restart Crowi ɾif you modified template ɾ Reload your browser

Slide 85

Slide 85 text

I want to add Some New feature

Slide 86

Slide 86 text

<$SPXJ> $MJFOU4JEF 4FSWFS4JEF Some new feature (server side) /PEFKT &YQSFTT .POHPPTF 4XJH #SPXTFS K2VFSZ 3FBDU #PPUTUSBQ

Slide 87

Slide 87 text

Keywords of using tools ɾ Express ɾ Mongoose ɾ Swig

Slide 88

Slide 88 text

MVC of Crowi .PEFM .POHPPTF 7JFX 4XJH3FBDU $POUSPMMFS &YQSFTT

Slide 89

Slide 89 text

$POUSPMMFS .PEFM Request flow SPVUFTJOEFYKT SPVUFT3065&@KT NPEFMT.0%&-@KT Request NPEFMT.0%&-@KT NPEFMT.0%&-@OKT SPVUFT3065&@KT SPVUFT3065&@OKT

Slide 90

Slide 90 text

Controller: Express

Slide 91

Slide 91 text

Express http://expressjs.com/

Slide 92

Slide 92 text

Go to directory $ cd crowi/lib/routes/

Slide 93

Slide 93 text

Modify files $ ls -1F
 admin.js
 attachment.js
 bookmark.js
 comment.js
 index.js
 installer.js
 login.js
 logout.js
 me.js
 page.js
 revision.js
 search.js
 user.js

Slide 94

Slide 94 text

Router: index.js

Slide 95

Slide 95 text

Router (index.js) var page = require('./page')(crowi, app); var middleware = require('../util/middlewares'); var loginRequired = middleware.loginRequired; app.get('/*/$', loginRequired(crowi, app), page.pageListShow); app.get('/*', loginRequired(crowi, app), page.pageShow); HTTP method Request path Router (Controller) method

Slide 96

Slide 96 text

page.pageShow (page.js) actions.pageShow = function(req, res) { var path = path || getPathFromRequest(req); Page.findPage(path, req.user, req.query.revision) .then(function(page) { // some routines return renderPage(page, req, res); }).catch(function(err) { // not found }); }; Page model method

Slide 97

Slide 97 text

Model: Mongoose

Slide 98

Slide 98 text

Mongoose http://mongoosejs.com/

Slide 99

Slide 99 text

Go to directory $ cd crowi/lib/models/

Slide 100

Slide 100 text

Modify files $ ls -1F
 attachment.js
 bookmark.js
 comment.js
 config.js
 index.js
 page.js
 revision.js
 updatePost.js
 user.js

Slide 101

Slide 101 text

DB Schema (page.js) pageSchema = new mongoose.Schema({ path: {type: String, required: true, index: true, unique: true}, revision: {type: ObjectId, ref: 'Revision'}, status: {type: String, default: STATUS_PUBLISHED, index: true}, creator: {type: ObjectId, ref: 'User', index: true}, createdAt: {type: Date, default: Date.now}, }

Slide 102

Slide 102 text

Sample Data

Slide 103

Slide 103 text

Sample page Page + Revision data

Slide 104

Slide 104 text

"path" field: String pageSchema = new mongoose.Schema({ path: {type: String, required: true, index: true, unique: true}, revision: {type: ObjectId, ref: 'Revision'}, status: {type: String, default: STATUS_PUBLISHED, index: true}, creator: {type: ObjectId, ref: 'User', index: true}, createdAt: {type: Date, default: Date.now}, } type: String

Slide 105

Slide 105 text

"path" field data { "_id": ObjectId("580fe42645419f6dcf5f4c49"), "path": "/user/suzuki/memo/2016/10/26/test_page", "revision": ObjectId("580fe42645419f6dcf5f4c4a") "status": "published", "creator": ObjectId("580401458462d38221a37094"), "createdAt": ISODate("2016-10-25T23:00:54.020Z"), } String data

Slide 106

Slide 106 text

"revision" field: ObjectId pageSchema = new mongoose.Schema({ path: {type: String, required: true, index: true, unique: true}, revision: {type: ObjectId, ref: 'Revision'}, status: {type: String, default: STATUS_PUBLISHED, index: true}, creator: {type: ObjectId, ref: 'User', index: true}, createdAt: {type: Date, default: Date.now}, } type: ObjectId ref: 'Revision'

Slide 107

Slide 107 text

"revision" field data { "_id": ObjectId("580fe42645419f6dcf5f4c49"), "path": "/user/suzuki/memo/2016/10/26/test_page", "revision": ObjectId("580fe42645419f6dcf5f4c4a") "status": "published", "creator": ObjectId("580401458462d38221a37094"), "createdAt": ISODate("2016-10-25T23:00:54.020Z"), } This means reference 'Revision'

Slide 108

Slide 108 text

Revision data { "_id": ObjectId("580fe42645419f6dcf5f4c4a"), "author": ObjectId("580401458462d38221a37094"), "body": "# test_page\r\n\r\nThis is a test page.\r\n", "path": "/user/suzuki/memo/2016/10/26/test_page", }

Slide 109

Slide 109 text

Schema types of Mongoose ɾ String ɾ Number ɾ Date ɾ Buffer ɾ Boolean ɾ Mixed ɾ ObjectId ɾ Array MongoDB is schema-less, But, Mongoose has schema types

Slide 110

Slide 110 text

Method (page.js) // find page and check if granted user pageSchema.statics.findPage = function(path, userData, revisionId, ignoreNotFound) { var self = this; return new Promise(function(resolve, reject) { self.findOne({path: path}, function(err, pageData) { if (err) { return reject(err); } if (pageData === null) { if (ignoreNotFound) { return resolve(null); } var pageNotFoundError = new Error('Page Not Found') pageNotFoundError.name = 'Crowi:Page:NotFound'; return reject(pageNotFoundError); } // snip

Slide 111

Slide 111 text

View: Swig + React

Slide 112

Slide 112 text

See CSS, Front-end JS page

Slide 113

Slide 113 text

How to debug

Slide 114

Slide 114 text

Debug log

Slide 115

Slide 115 text

var debug = require('debug')('crowi:routes:page'); Define debug (routes/page.js) Key name

Slide 116

Slide 116 text

actions.pageListShow = function(req, res) { var path = getPathFromRequest(req); debug('Page list show', path); Call debug (routes/page.js) No output by default

Slide 117

Slide 117 text

Run Crowi with DEBUG environment 
 $ PASSWORD_SEED=somesecretstring \
 MONGO_URI=mongodb://localhost/crowi \
 ELASTICSEARCH_URI=localhost:9200 \
 REDIS_URL=localhost \
 FILE_UPLOAD=local \
 DEBUG=crowi:routes:page \
 node app.js Set DEBUG env.

Slide 118

Slide 118 text

Run Crowi with DEBUG environment [development] Express server listening on port 3000
 crowi:routes:page Page list show +0ms / Debug log

Slide 119

Slide 119 text

lib/routes/page.js: var debug = require('debug')('crowi:routes:page'); lib/routes/revision.js: var debug = require('debug')('crowi:routes:revision'); lib/models/page.js: var debug = require('debug')('crowi:models:page'); lib/models/comment.js: var debug = require('debug')('crowi:models:comment'); Wildcard DEBUG env. DEBUG=crowi:models:*

Slide 120

Slide 120 text

More information about DEBUG ɾ Debugging Express ɾ http://expressjs.com/en/guide/debugging.html (English) ɾ http://expressjs.com/ja/guide/debugging.html (Japanese)

Slide 121

Slide 121 text

View data

Slide 122

Slide 122 text

Get container name MondoDB container name is here. For this example, "crowidevelop_mongodb_1"

Slide 123

Slide 123 text

Login container $ docker exec -it crowidevelop_mongodb_1 /bin/bash Execute "/bin/bash" in container

Slide 124

Slide 124 text

Run mongo CLI root@f7e7c8334c37:/# mongo
 MongoDB shell version: 3.2.9
 connecting to: local
 >

Slide 125

Slide 125 text

Select database > use crowi
 switched to db crowi

Slide 126

Slide 126 text

Show collections (SHOW TABLES) > show collections
 attachments
 bookmarks
 comments
 configs
 pages
 revisions
 updateposts
 users Collections are mapped to Models

Slide 127

Slide 127 text

Find data (SELECT data) > db.pages.find(); SELECT * FROM pages;

Slide 128

Slide 128 text

Find data: WHERE Clause > db.pages.find({"_id": ObjectId("580fe42645419f6dcf5f4c49")}); SELECT * FROM pages WHERE _id = ObjectId("580fe42645419f6dcf5f4c49")

Slide 129

Slide 129 text

find() > db.pages.find();
 { "_id" : ObjectId("580fe42645419f6dcf5f4c49"), "redirectTo" : null, "updatedAt" : ISODate("2016-10-25T23:00:54.091Z"), "lastUpdateUser" : ObjectId("580401458462d38221a37094"), "creator" : ObjectId("580401458462d38221a37094"), "path" : "/ user/suzuki/memo/2016/10/26/test_page", "createdAt" : ISODate("2016-10-25T23:00:54.020Z"), "extended" : "{}", "commentCount" : 1, "seenUsers" : [ ObjectId("580401458462d38221a37094") ], "liker" : [ ], "grantedUsers" : [ ObjectId("580401458462d38221a37094") ], "grant" : 1, "status" : "published", "__v" : 1, "revision" : ObjectId("580fe42645419f6dcf5f4c4a") }

Slide 130

Slide 130 text

find().pretty() > db.pages.find().pretty();
 {
 "_id" : ObjectId("580fe42645419f6dcf5f4c49"),
 "redirectTo" : null,
 "updatedAt" : ISODate("2016-10-25T23:00:54.091Z"),
 "lastUpdateUser" : ObjectId("580401458462d38221a37094"),
 "creator" : ObjectId("580401458462d38221a37094"),
 "path" : "/user/suzuki/memo/2016/10/26/test_page",
 "createdAt" : ISODate("2016-10-25T23:00:54.020Z"),
 "extended" : "{}",
 "commentCount" : 1,
 // snip
 "status" : "published",
 "revision" : ObjectId("580fe42645419f6dcf5f4c4a")
 }

Slide 131

Slide 131 text

More information about MongoDB ɾ The mongo Shell ɾ https://docs.mongodb.com/v3.2/mongo/ ɾ GUI ɾ https://docs.mongodb.com/ecosystem/tools/administration-interfaces/

Slide 132

Slide 132 text

How to Test?

Slide 133

Slide 133 text

Run tests $ ./node_modules/.bin/gulp test OR $ MONGO_URI=localhost:crowi_test \
 ./node_modules/.bin/gulp test "crowi_test" DB is used "crowi" (default) DB is used I recommend to use different name from "crowi"

Slide 134

Slide 134 text

Files $ cd crowi/tests/ $ ls -1F
 bootstrap.js
 crowi/
 models/
 util/
 utils.js

Slide 135

Slide 135 text

Problems ɾ Coverage ɾ We have little tests... ɾ No View test ɾ React component tests? ɾ E2E tests?

Slide 136

Slide 136 text

Conclusion

Slide 137

Slide 137 text

Crowi is a growing software ɾ Crowi is very simple and useful ɾ Growing software ɾ We are using the Crowi in our company ɾ When received feedback, 
 we will fixed that problem or create new features ɾ Try to use the Crowi in your organization

Slide 138

Slide 138 text

We learned first-step to develop Crowi ɾ Today, we have learned how to develop the Crowi ɾ You can use your experience in PHP ɾ It is not difficult to develop

Slide 139

Slide 139 text

Today's theme http://phpcon.php.gr.jp/2016/

Slide 140

Slide 140 text

Next Action: Contribute! ɾ Code for Crowi ɾ Pull Request, Issue, Document, etc. ɾ Talk about Crowi ɾ Gitter, Twitter, Facebook, Blog, etc. ɾ #crowi in Twitter ɾ Gitter: https://gitter.im/crowi/general

Slide 141

Slide 141 text

And, we are hiring! https://www.mercari.com/jp/jobs/ But, a full-time Crowi job is not exist

Slide 142

Slide 142 text

Thank you

Slide 143

Slide 143 text

Appendix

Slide 144

Slide 144 text

Appendix ɾ Crowi ɾ https://github.com/crowi/crowi ɾ Crowi develop ɾ https://github.com/suzuki/crowi-develop ɾ Crowi book (blog) ɾ https://medium.com/crowi-book ɾ Crowi gitter ɾ https://gitter.im/crowi/general