• R/O
  • HTTP
  • SSH
  • HTTPS

grid-chef-repo: コミット

Grid環境構築用のChefリポジトリです。


コミットメタ情報

リビジョン17ab58a26b413089e057014d1c743d20d410b2e0 (tree)
日時2018-02-03 16:46:53
作者whitestar <whitestar@user...>
コミッターwhitestar

ログメッセージ

adds MySQL support.

変更サマリ

差分

--- a/cookbooks/screwdriver/CHANGELOG.md
+++ b/cookbooks/screwdriver/CHANGELOG.md
@@ -1,5 +1,9 @@
11 # screwdriver CHANGELOG
22
3+0.4.0
4+-----
5+- adds MySQL support.
6+
37 0.3.1
48 -----
59 - revises documents.
--- a/cookbooks/screwdriver/README.md
+++ b/cookbooks/screwdriver/README.md
@@ -19,8 +19,12 @@ This cookbook sets up a Screwdriver CI/CD service by Docker Compose.
1919 - [JWT private and public keys management by Chef Vault](#jwt-private-and-public-keys-management-by-chef-vault)
2020 - [Cookie password management by Chef Vault](#cookie-password-management-by-chef-vault)
2121 - [Secrets encryption password management by Chef Vault](#secrets-encryption-password-management-by-chef-vault)
22- - [OAuth client ID and secret management by Chef Vault](#oauth-client-id-and-secret-management-by-chef-vault)
23- - [GitHub webhook secret management by Chef Vault](#github-webhook-secret-management-by-chef-vault)
22+ - [Database username management (for MySQL, PostgreSQL,...) by Chef Vault](#database-username-management-for-mysql-postgresql-by-chef-vault)
23+ - [Database password management (for MySQL, PostgreSQL,...) by Chef Vault](#database-password-management-for-mysql-postgresql-by-chef-vault)
24+ - [Database root password management (for MySQL, PostgreSQL,...) by Chef Vault](#database-root-password-management-for-mysql-postgresql-by-chef-vault)
25+ - [OAuth client ID, secret and GitHub webhook secret management by Chef Vault](#oauth-client-id-secret-and-github-webhook-secret-management-by-chef-vault)
26+ - [Note](#note)
27+ - [Database Initialization](#database-initialization)
2428 - [License and Authors](#license-and-authors)
2529
2630 ## Requirements
@@ -48,6 +52,9 @@ This cookbook sets up a Screwdriver CI/CD service by Docker Compose.
4852 |`['screwdriver']['jwt_public_key_vault_item']`|Hash|Optional, Sets a JWT public key from Chef Vault. See `attributes/default.rb`|`{}`|
4953 |`['screwdriver']['cookie_password_vault_item']`|Hash|Optional, Sets a session cookie password from Chef Vault. See `attributes/default.rb`|`{}`|
5054 |`['screwdriver']['password_vault_item']`|Hash|Optional, Sets a password for secrets encryption from Chef Vault. See `attributes/default.rb`|`{}`|
55+|`['screwdriver']['db_username_vault_item']`|Hash|Optional, Sets a database username from Chef Vault. See `attributes/default.rb`|`{}`|
56+|`['screwdriver']['db_password_vault_item']`|Hash|Optional, Sets a database password from Chef Vault. See `attributes/default.rb`|`{}`|
57+|`['screwdriver']['db_root_password_vault_item']`|Hash|Optional, Sets a database password for the root user from Chef Vault. See `attributes/default.rb`|`{}`|
5158 |`['screwdriver']['ui']['tls_setup_mode']`|String|`'reverseproxy'` only. Note: [_Add TLS support to UI docker container #377_](https://github.com/screwdriver-cd/screwdriver/issues/377)|`'reverseproxy'`|
5259 |`['screwdriver']['api']['config']`|Hash|This hash object is expanded to a `/config/local.yaml` file in the API Docker container.|See `attributes/default.rb`|
5360 |`['screwdriver']['api']['scms_vault_items']`|Hash|This hash contains Chef Vault item definitions of SCM's secrets.|See `attributes/default.rb`|
@@ -315,18 +322,18 @@ override_attributes(
315322
316323 ```text
317324 $ ruby -rjson -e 'puts JSON.generate({"private" => File.read("screwdriver.io.example.com.prod.key")})' \
318-> > ~/tmp/screwdriver.io.example.com.prod.key.json
325+> > ~/sec/tmp/screwdriver.io.example.com.prod.key.json
319326
320327 $ ruby -rjson -e 'puts JSON.generate({"public" => File.read("screwdriver.io.example.com.prod.crt")})' \
321-> > ~/tmp/screwdriver.io.example.com.prod.crt.json
328+> > ~/sec/tmp/screwdriver.io.example.com.prod.crt.json
322329
323330 $ cd $CHEF_REPO_PATH
324331
325332 $ knife vault create ssl_server_keys screwdriver.io.example.com.prod \
326-> --json ~/tmp/screwdriver.io.example.com.prod.key.json
333+> --json ~/sec/tmp/screwdriver.io.example.com.prod.key.json
327334
328335 $ knife vault create ssl_server_certs screwdriver.io.example.com.prod \
329-> --json ~/tmp/screwdriver.io.example.com.prod.crt.json
336+> --json ~/sec/tmp/screwdriver.io.example.com.prod.crt.json
330337 ```
331338
332339 - grant reference permission to the screwdriver host
@@ -411,7 +418,7 @@ override_attributes(
411418
412419 ```text
413420 # A password used for encrypting session data. Needs to be minimum 32 characters
414-$ cat ~/tmp/screwdriver_cookie_password.json
421+$ cat ~/sec/tmp/screwdriver_cookie_password.json
415422 {"password":"********************************"}
416423
417424 $ cd $CHEF_REPO_PATH
@@ -447,7 +454,7 @@ override_attributes(
447454
448455 ```text
449456 # A password used for encrypting stored secrets. Needs to be minimum 32 characters
450-$ cat ~/tmp/screwdriver_password.json
457+$ cat ~/sec/tmp/screwdriver_password.json
451458 {"password":"********************************"}
452459
453460 $ cd $CHEF_REPO_PATH
@@ -477,6 +484,111 @@ override_attributes(
477484 )
478485 ```
479486
487+### Database username management (for MySQL, PostgreSQL,...) by Chef Vault
488+
489+- create vault items.
490+
491+```text
492+$ cat ~/sec/tmp/screwdriver_db_username.json
493+{"username":"********************************"}
494+
495+$ cd $CHEF_REPO_PATH
496+$ knife vault create screwdriver db_username --json ~/sec/tmp/screwdriver_db_username.json
497+```
498+
499+- grant reference permission to the screwdriver host
500+
501+```text
502+$ knife vault update screwdriver db_username -S 'name:screwdriver-host.example.com'
503+```
504+
505+- modify attributes
506+
507+```ruby
508+override_attributes(
509+ 'screwdriver' => {
510+ # ...
511+ 'db_username_vault_item' => {
512+ 'vault' => 'screwdriver',
513+ 'name' => 'db_username',
514+ 'env_context' => false,
515+ 'key' => 'username',
516+ },
517+ # ...
518+ },
519+)
520+```
521+
522+### Database password management (for MySQL, PostgreSQL,...) by Chef Vault
523+
524+- create vault items.
525+
526+```text
527+$ cat ~/sec/tmp/screwdriver_db_password.json
528+{"password":"********************************"}
529+
530+$ cd $CHEF_REPO_PATH
531+$ knife vault create screwdriver db_password --json ~/sec/tmp/screwdriver_db_password.json
532+```
533+
534+- grant reference permission to the screwdriver host
535+
536+```text
537+$ knife vault update screwdriver db_password -S 'name:screwdriver-host.example.com'
538+```
539+
540+- modify attributes
541+
542+```ruby
543+override_attributes(
544+ 'screwdriver' => {
545+ # ...
546+ 'db_password_vault_item' => {
547+ 'vault' => 'screwdriver',
548+ 'name' => 'db_password',
549+ 'env_context' => false,
550+ 'key' => 'password',
551+ },
552+ # ...
553+ },
554+)
555+```
556+
557+### Database root password management (for MySQL, PostgreSQL,...) by Chef Vault
558+
559+- create vault items.
560+
561+```text
562+$ cat ~/sec/tmp/screwdriver_db_root_password.json
563+{"password":"********************************"}
564+
565+$ cd $CHEF_REPO_PATH
566+$ knife vault create screwdriver db_root_password --json ~/sec/tmp/screwdriver_db_root_password.json
567+```
568+
569+- grant reference permission to the screwdriver host
570+
571+```text
572+$ knife vault update screwdriver db_root_password -S 'name:screwdriver-host.example.com'
573+```
574+
575+- modify attributes
576+
577+```ruby
578+override_attributes(
579+ 'screwdriver' => {
580+ # ...
581+ 'db_root_password_vault_item' => {
582+ 'vault' => 'screwdriver',
583+ 'name' => 'db_root_password',
584+ 'env_context' => false,
585+ 'key' => 'password',
586+ },
587+ # ...
588+ },
589+)
590+```
591+
480592 ### OAuth client ID, secret and GitHub webhook secret management by Chef Vault
481593
482594 - create vault items.
@@ -538,6 +650,25 @@ override_attributes(
538650 )
539651 ```
540652
653+### Note
654+
655+#### Database Initialization
656+
657+If you use database other than sqlite, its database initialization will takes a few tens of seconds.
658+You should run a database container only at the beginning and then start the others.
659+```
660+$ sudo docker-compose up -d db
661+...
662+Creating network "screwdriver_default" with the default driver
663+Creating screwdriver_db_1 ... done
664+
665+$ sudo docker-compose up -d
666+screwdriver_db_1 is up-to-date
667+Creating screwdriver_api_1 ... done
668+Creating screwdriver_ui_1 ... done
669+Creating screwdriver_store_1 ... done
670+```
671+
541672 ## License and Authors
542673
543674 - Author:: whitestar at osdn.jp
--- a/cookbooks/screwdriver/attributes/default.rb
+++ b/cookbooks/screwdriver/attributes/default.rb
@@ -76,6 +76,42 @@ default['screwdriver']['password_vault_item'] = {
7676 #'key' => 'hash/path/to/password', # real hash path: "/#{node.chef_environment}/hash/path/to/password"
7777 =end
7878 }
79+default['screwdriver']['db_username_vault_item'] = {
80+=begin
81+ 'vault' => 'screwdriver',
82+ 'name' => 'db_username',
83+ # single usernaem or nested hash username path delimited by slash
84+ 'env_context' => false,
85+ 'key' => 'username', # real hash path: "/username"
86+ # or nested hash password path delimited by slash
87+ #'env_context' => true,
88+ #'key' => 'hash/path/to/username', # real hash path: "/#{node.chef_environment}/hash/path/to/username"
89+=end
90+}
91+default['screwdriver']['db_password_vault_item'] = {
92+=begin
93+ 'vault' => 'screwdriver',
94+ 'name' => 'db_password',
95+ # single password or nested hash password path delimited by slash
96+ 'env_context' => false,
97+ 'key' => 'password', # real hash path: "/password"
98+ # or nested hash password path delimited by slash
99+ #'env_context' => true,
100+ #'key' => 'hash/path/to/password', # real hash path: "/#{node.chef_environment}/hash/path/to/password"
101+=end
102+}
103+default['screwdriver']['db_root_password_vault_item'] = {
104+=begin
105+ 'vault' => 'screwdriver',
106+ 'name' => 'db_root_password',
107+ # single password or nested hash password path delimited by slash
108+ 'env_context' => false,
109+ 'key' => 'password', # real hash path: "/password"
110+ # or nested hash password path delimited by slash
111+ #'env_context' => true,
112+ #'key' => 'hash/path/to/password', # real hash path: "/#{node.chef_environment}/hash/path/to/password"
113+=end
114+}
79115
80116 force_override['screwdriver']['ui']['tls_setup_mode'] = 'reverseproxy'
81117 # These hash objects are expanded to a `/config/local.yaml` file in each Docker container.
@@ -260,7 +296,7 @@ version_2_config = {
260296 'volumes' => [
261297 '/var/run/docker.sock:/var/run/docker.sock:rw',
262298 # This volume will be set by the screwdriver::docker-compose recipe automatically.
263- #"#{node['screwdriver']['docker-compose']['data_dir']}:/sd-data:rw",
299+ #"#{node['screwdriver']['docker-compose']['data_dir']}:/sd-data:rw", # for sqlite
264300 ],
265301 'environment' => {
266302 # See:
@@ -279,8 +315,16 @@ version_2_config = {
279315 'SECRET_WHITELIST' => '[]',
280316 'SECRET_ADMINS' => '[]',
281317 'DATASTORE_PLUGIN' => 'sequelize',
318+ 'DATASTORE_SEQUELIZE_DATABASE' => 'screwdriver',
282319 'DATASTORE_SEQUELIZE_DIALECT' => 'sqlite',
283- 'DATASTORE_SEQUELIZE_STORAGE' => '/sd-data/storage.db',
320+ # This variable will be set by the screwdriver::docker-compose recipe automatically.
321+ #'DATASTORE_SEQUELIZE_STORAGE' => '/sd-data/storage.db',
322+ # for MySQL
323+ #'DATASTORE_SEQUELIZE_DIALECT' => 'mysql',
324+ # These variables will be set by the screwdriver::docker-compose recipe automatically.
325+ #'DATASTORE_SEQUELIZE_USERNAME' => '${DB_USERNAME}',
326+ #'DATASTORE_SEQUELIZE_PASSWORD' => '${DB_PASSWORD}',
327+ #'DATASTORE_SEQUELIZE_HOST' => 'db',
284328 # This variable will be set by the screwdriver::docker-compose recipe automatically.
285329 #'IS_HTTPS' => 'false',
286330 #'NODE_TLS_REJECT_UNAUTHORIZED' => '0', # workaround for self-signed cetificates
@@ -357,4 +401,23 @@ EOS
357401 },
358402 }
359403
404+config_srvs = node['screwdriver']['docker-compose']['config']['services']
405+case config_srvs['api']['environment']['DATASTORE_SEQUELIZE_DIALECT']
406+when 'mysql'
407+ version_2_config['services']['db'] = {
408+ 'image' => 'mysql:5',
409+ 'volumes' => [
410+ # This variable will be set by the screwdriver::docker-compose recipe automatically.
411+ #"#{node['screwdriver']['docker-compose']['data_dir']}/mysql:/var/lib/mysql:rw",
412+ ],
413+ 'environment' => {
414+ # These variables will be set by the screwdriver::docker-compose recipe automatically.
415+ #'MYSQL_ROOT_PASSWORD' => '${DB_ROOT_PASSWORD}',
416+ #'MYSQL_USER' => '${DB_USERNAME}',
417+ #'MYSQL_PASSWORD' => '${DB_PASSWORD}',
418+ #'MYSQL_DATABASE' => 'screwdriver',
419+ },
420+ }
421+end
422+
360423 default['screwdriver']['docker-compose']['config'] = version_2_config
--- a/cookbooks/screwdriver/recipes/docker-compose.rb
+++ b/cookbooks/screwdriver/recipes/docker-compose.rb
@@ -111,7 +111,6 @@ else
111111 api_port = (elms.size == 2 ? elms[0] : elms[1]) if elms.last == api_in_port
112112 }
113113 end
114-api_vols.push("#{data_dir}:/sd-data:rw")
115114
116115 override_api_config['executor'] = default_executor if override_api_config['executor'].empty?
117116
@@ -197,7 +196,7 @@ node['screwdriver']['api']['scms_vault_items'].each {|scm, props|
197196 }
198197 }
199198 =begin
200-# DEPRECATED!!
199+# **DEPRECATED!!**
201200 oauth_client_id = nil
202201 oauth_client_id_vault_item = node['screwdriver']['docker-compose']['oauth_client_id_vault_item']
203202 unless oauth_client_id_vault_item.empty?
@@ -220,6 +219,60 @@ unless webhook_github_secret_vault_item.empty?
220219 end
221220 =end
222221
222+db_username = nil
223+db_username_vault_item = node['screwdriver']['db_username_vault_item']
224+unless db_username_vault_item.empty?
225+ db_username = get_vault_item_value(db_username_vault_item)
226+ api_envs['DATASTORE_SEQUELIZE_USERNAME'] = '${DB_USERNAME}'
227+end
228+
229+db_password = nil
230+db_password_vault_item = node['screwdriver']['db_password_vault_item']
231+unless db_password_vault_item.empty?
232+ db_password = get_vault_item_value(db_password_vault_item)
233+ api_envs['DATASTORE_SEQUELIZE_PASSWORD'] = '${DB_PASSWORD}'
234+end
235+
236+db_root_password = nil
237+db_root_password_vault_item = node['screwdriver']['db_root_password_vault_item']
238+unless db_root_password_vault_item.empty?
239+ db_root_password = get_vault_item_value(db_root_password_vault_item)
240+end
241+
242+db_dialect = api_envs_org['DATASTORE_SEQUELIZE_DIALECT']
243+case db_dialect
244+when 'sqlite'
245+ api_vols.push("#{data_dir}:/sd-data:rw")
246+ api_envs['DATASTORE_SEQUELIZE_STORAGE'] = '/sd-data/storage.db'
247+when 'mysql'
248+ override_config_srvs['api']['links'] = ['db']
249+ api_envs['DATASTORE_SEQUELIZE_HOST'] = 'db'
250+end
251+
252+# db
253+if db_dialect != 'sqlite'
254+ #db_envs_org = config_srvs['db']['environment']
255+ db_envs = {}
256+ db_vols = config_srvs['db']['volumes'].to_a
257+
258+ case db_dialect
259+ when 'mysql'
260+ mysql_data_dir = "#{data_dir}/mysql"
261+ resources(directory: mysql_data_dir) rescue directory mysql_data_dir do
262+ owner 'root'
263+ group 'root'
264+ mode '0755'
265+ recursive true
266+ end
267+
268+ db_envs['MYSQL_DATABASE'] = api_envs_org['DATASTORE_SEQUELIZE_DATABASE']
269+ db_envs['MYSQL_USER'] = '${DB_USERNAME}' unless db_username.nil?
270+ db_envs['MYSQL_PASSWORD'] = '${DB_PASSWORD}' unless db_password.nil?
271+ db_envs['MYSQL_ROOT_PASSWORD'] = '${DB_ROOT_PASSWORD}' unless db_root_password.nil?
272+ db_vols.push("#{mysql_data_dir}:/var/lib/mysql:rw")
273+ end
274+end
275+
223276 # ui
224277 #ui_envs_org = config_srvs['ui']['environment']
225278 ui_envs = {}
@@ -397,10 +450,16 @@ end
397450 force_override_config_srvs['api']['environment'] = api_envs unless api_envs.empty?
398451 force_override_config_srvs['ui']['environment'] = ui_envs unless ui_envs.empty?
399452 force_override_config_srvs['store']['environment'] = store_envs unless store_envs.empty?
453+if db_dialect != 'sqlite'
454+ force_override_config_srvs['db']['environment'] = db_envs unless db_envs.empty?
455+end
400456 # reset vlumes array.
401457 override_config_srvs['api']['volumes'] = api_vols unless api_vols.empty?
402458 override_config_srvs['ui']['volumes'] = ui_vols unless ui_vols.empty?
403459 override_config_srvs['store']['volumes'] = store_vols unless store_vols.empty?
460+if db_dialect != 'sqlite'
461+ override_config_srvs['db']['volumes'] = db_vols unless db_vols.empty?
462+end
404463
405464 template env_file do
406465 source 'opt/docker-compose/app/screwdriver/.env'
@@ -411,11 +470,15 @@ template env_file do
411470 # prevent Chef from logging password attribute value.
412471 variables(
413472 # secrets
473+ cookie_password: cookie_password,
474+ password: password,
475+ db_username: db_username,
476+ db_password: db_password,
477+ db_root_password: db_root_password,
478+ # **DEPRECATED!!**
414479 # JWT keys setting -> /config/local.yaml
415480 #jwt_private_key: jwt_private_key,
416481 #jwt_public_key: jwt_public_key,
417- cookie_password: cookie_password,
418- password: password,
419482 # SCM secrets setting -> /config/local.yaml
420483 #oauth_client_id: oauth_client_id,
421484 #oauth_client_secret: oauth_client_secret,
--- a/cookbooks/screwdriver/templates/default/opt/docker-compose/app/screwdriver/.env
+++ b/cookbooks/screwdriver/templates/default/opt/docker-compose/app/screwdriver/.env
@@ -24,3 +24,12 @@ SECRET_OAUTH_CLIENT_SECRET=<%= @oauth_client_secret %>
2424 <% unless @webhook_github_secret.nil? %>
2525 WEBHOOK_GITHUB_SECRET=<%= @webhook_github_secret %>
2626 <% end %>
27+<% unless @db_username.nil? %>
28+DB_USERNAME=<%= @db_username %>
29+<% end %>
30+<% unless @db_password.nil? %>
31+DB_PASSWORD=<%= @db_password %>
32+<% end %>
33+<% unless @db_root_password.nil? %>
34+DB_ROOT_PASSWORD=<%= @db_root_password %>
35+<% end %>
--- a/cookbooks/screwdriver/version
+++ b/cookbooks/screwdriver/version
@@ -1 +1 @@
1-0.3.1
1+0.4.0
旧リポジトリブラウザで表示