Standalone Neutron: How we integrated it with Abiquo

This is a talk I gave at the OpenStack BCN 7th birthday meetup.
In it I explain how we integrated a standalone Neutron with Abiquo to implement an SDN solution w/o a full OpenStack.

Transcript

1. Standalone Neutron How we integrated it with Abiquo Ignasi Barrera Senior So-ware Engineer, Abiquo Apache So-ware Founda:on Member
2. The Abiquo use case 20 July 2017 OpenStack BCN 7th birthday meetup 2 Abiquo had a basic self-service networking model for KVM based on VLANs. We wanted to add support for: •  Firewalls •  Load Balancers •  Floa:ng Ips •  Other networking technologies (not a priority)
3. Available op:ons 20 July 2017 OpenStack BCN 7th birthday meetup 3 •  Implement our own solu:on •  Explore the exis:ng SDN products – Highly customizable – Mul:-tenant – Open source – Good community support
4. Choosing OpenStack 20 July 2017 OpenStack BCN 7th birthday meetup 4 OpenStack already had what we needed, but we only needed some parts of the relevant compute stack: •  Keystone – For access control and mul:- tenancy •  Neutron – To provide the SDN features •  Nova – We already had our own hypervisor agent and wanted to integrate it with Neutron
5. Neutron implementa:on 20 July 2017 OpenStack BCN 7th birthday meetup 5 Our hypervisor agent already managed VLANs and Linux bridges, so we started using: •  VLANs for user defined networks •  The Linux bridges plugin •  Iptables for security groups •  The LBaaS API with HAProxy for load balancers
6. Neutron with Linux bridges 20 July 2017 OpenStack BCN 7th birthday meetup 6
7. The controller node 20 July 2017 OpenStack BCN 7th birthday meetup 7 The controller node keeps the logical networking model. We directly manage it using the Neutron API to: •  Create networks and subnets •  Manage security groups •  Create and configure ports •  Manage floa:ng IPs •  Manage load balancers
8. The compute node 20 July 2017 OpenStack BCN 7th birthday meetup 8
9. The compute agent 20 July 2017 OpenStack BCN 7th birthday meetup 9 We will rely on the Linux bridges agent to create all stuff
10. The compute agent 20 July 2017 OpenStack BCN 7th birthday meetup 10 We will rely on the Linux bridges agent to create all stuff What needs to be done to let the agent know that the infrastructure is ready?
11. The compute agent 20 July 2017 OpenStack BCN 7th birthday meetup 11 We will rely on the Linux bridges agent to create all stuff What needs to be done to let the agent know that the infrastructure is ready? How do we make the ports ACTIVE?
12. Example: VM with a single IP and SSH access 20 July 2017 OpenStack BCN 7th birthday meetup 12 Create the network (neutron) net-create test --tenant-id b7de212040bc4518b9fe11b53e8cbed7 Created a new network: +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | True | | id | 1a817eb7-ef09-455a-95dc-75e24d7b3d13 | | mtu | 0 | | name | test | | provider:network_type | vlan | | provider:physical_network | abq-vlans | | provider:segmentation_id | 78 | | router:external | False | | shared | False | | status | ACTIVE | | subnets | | | tenant_id | b7de212040bc4518b9fe11b53e8cbed7 | +---------------------------+--------------------------------------+
13. Example: VM with a single IP and SSH access 20 July 2017 OpenStack BCN 7th birthday meetup 13 Create the subnet (neutron) subnet-create test 192.168.1.0/24 --name test-subnet --enable-dhcp --tenant-id b7de212040bc4518b9fe11b53e8cbed7 Created a new subnet: +-------------------+--------------------------------------------------+ | Field | Value | +-------------------+--------------------------------------------------+ | allocation_pools | {"start": "192.168.1.2", "end": "192.168.1.254"} | | cidr | 192.168.1.0/24 | | dns_nameservers | | | enable_dhcp | True | | gateway_ip | 192.168.1.1 | | host_routes | | | id | c108f047-22f8-425c-9b33-305e74cc900e | | ip_version | 4 | | ipv6_address_mode | | | ipv6_ra_mode | | | name | test-subnet | | network_id | 1a817eb7-ef09-455a-95dc-75e24d7b3d13 | | subnetpool_id | | | tenant_id | b7de212040bc4518b9fe11b53e8cbed7 | +-------------------+--------------------------------------------------+
14. Example: VM with a single IP and SSH access 20 July 2017 OpenStack BCN 7th birthday meetup 14 Create the tenant router (neutron) router-create test-router --tenant-id b7de212040bc4518b9fe11b53e8cbed7 Created a new router: +-----------------------+--------------------------------------+ | Field | Value | +-----------------------+--------------------------------------+ | admin_state_up | True | | distributed | False | | external_gateway_info | | | ha | False | | id | 7983a549-a08e-4928-9614-d06c13ef1cb3 | | name | test-router | | routes | | | status | ACTIVE | | tenant_id | b7de212040bc4518b9fe11b53e8cbed7 | +-----------------------+--------------------------------------+ (neutron) router-gateway-set test-router abq-external Set gateway for router test-router (neutron) router-interface-add test-router test-subnet Added interface 75809387-a8bb-4db4-a39e-8064e47927cc to router test-router.
15. Example: VM with a single IP and SSH access 20 July 2017 OpenStack BCN 7th birthday meetup 15 Create the security group (neutron) security-group-create test-sg --tenant-id b7de212040bc4518b9fe11b53e8cbed7 (neutron) security-group-rule-create --protocol tcp --direction ingress --port-range-min 22 --port-range-max 22 --tenant-id b7de212040bc4518b9fe11b53e8cbed7 test-sg Created a new security_group_rule: +-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | direction | ingress | | ethertype | IPv4 | | id | e0358073-0aa4-442a-8276-c91b05e7cbff | | port_range_max | 22 | | port_range_min | 22 | | protocol | tcp | | remote_group_id | | | remote_ip_prefix | | | security_group_id | 9105859e-1838-416a-87b7-33b764785d9e | | tenant_id | b7de212040bc4518b9fe11b53e8cbed7 | +-------------------+--------------------------------------+
16. Example: VM with a single IP and SSH access 20 July 2017 OpenStack BCN 7th birthday meetup 16 Create the port (neutron) port-create --name test-port --device-owner compute:Abiquo --security-group test-sg --tenant-id b7de212040bc4518b9fe11b53e8cbed7 test Created a new port: +-----------------------+------------------------------------------------------------------------------------+ | Field | Value | +-----------------------+------------------------------------------------------------------------------------+ | admin_state_up | True | | allowed_address_pairs | | | binding:host_id | | | binding:profile | {} | | binding:vif_details | {} | | binding:vif_type | unbound | | binding:vnic_type | normal | | device_id | | | device_owner | compute:Abiquo | | fixed_ips | {"subnet_id": "c108f047-22f8-425c-9b33-305e74cc900e", "ip_address": "192.168.1.3"} | | id | 18365a93-63a5-4212-a23f-60adbff44024 | | mac_address | fa:16:3e:ff:e2:04 | | name | test-port | | network_id | 1a817eb7-ef09-455a-95dc-75e24d7b3d13 | | security_groups | 9105859e-1838-416a-87b7-33b764785d9e | | status | DOWN | | tenant_id | b7de212040bc4518b9fe11b53e8cbed7 | +-----------------------+------------------------------------------------------------------------------------+
17. Example: VM with a single IP and SSH access 20 July 2017 OpenStack BCN 7th birthday meetup 17 Create the port (neutron) port-create --name test-port --device-owner compute:Abiquo --security-group test-sg --tenant-id b7de212040bc4518b9fe11b53e8cbed7 test Created a new port: +-----------------------+------------------------------------------------------------------------------------+ | Field | Value | +-----------------------+------------------------------------------------------------------------------------+ | admin_state_up | True | | allowed_address_pairs | | | binding:host_id | | | binding:profile | {} | | binding:vif_details | {} | | binding:vif_type | unbound | | binding:vnic_type | normal | | device_id | | | device_owner | compute:Abiquo | | fixed_ips | {"subnet_id": "c108f047-22f8-425c-9b33-305e74cc900e", "ip_address": "192.168.1.3"} | | id | 18365a93-63a5-4212-a23f-60adbff44024 | | mac_address | fa:16:3e:ff:e2:04 | | name | test-port | | network_id | 1a817eb7-ef09-455a-95dc-75e24d7b3d13 | | security_groups | 9105859e-1838-416a-87b7-33b764785d9e | | status | DOWN | | tenant_id | b7de212040bc4518b9fe11b53e8cbed7 | +-----------------------+------------------------------------------------------------------------------------+
18. Example: VM with a single IP and SSH access 20 July 2017 OpenStack BCN 7th birthday meetup 18 Tell Neutron where the port is depoyed (neutron) agent-list -c agent_type -c host -c alive +--------------------+----------+-------+ | agent_type | host | alive | +--------------------+----------+-------+ | Linux bridge agent | neutron | :-) | | L3 agent | neutron | :-) | | Metadata agent | neutron | :-) | | Linux bridge agent | nacx-kvm | :-) | | Loadbalancer agent | neutron | :-) | | DHCP agent | neutron | :-) | +--------------------+----------+-------+
19. Example: VM with a single IP and SSH access 20 July 2017 OpenStack BCN 7th birthday meetup 19 Tell Neutron where the port is depoyed (neutron) port-update --binding:host_id=nacx-kvm test-port Updated port: test-port (neutron) port-show test-port +-----------------------+------------------------------------------------------------------------------------+ | Field | Value | +-----------------------+------------------------------------------------------------------------------------+ | admin_state_up | True | | allowed_address_pairs | | | binding:host_id | nacx-kvm | | binding:profile | {} | | binding:vif_details | {"port_filter": true} | | binding:vif_type | bridge | | binding:vnic_type | normal | | device_id | | | device_owner | compute:Abiquo | | extra_dhcp_opts | | | fixed_ips | {"subnet_id": "c108f047-22f8-425c-9b33-305e74cc900e", "ip_address": "192.168.1.3"} | | id | 18365a93-63a5-4212-a23f-60adbff44024 | | mac_address | fa:16:3e:ff:e2:04 | | name | test-port | | network_id | 1a817eb7-ef09-455a-95dc-75e24d7b3d13 | | security_groups | 9105859e-1838-416a-87b7-33b764785d9e | | status | DOWN | | tenant_id | b7de212040bc4518b9fe11b53e8cbed7 | +-----------------------+------------------------------------------------------------------------------------+
20. Example: VM with a single IP and SSH access 20 July 2017 OpenStack BCN 7th birthday meetup 20 Tell Neutron where the port is depoyed (neutron) port-update --binding:host_id=nacx-kvm test-port Updated port: test-port (neutron) port-show test-port +-----------------------+------------------------------------------------------------------------------------+ | Field | Value | +-----------------------+------------------------------------------------------------------------------------+ | admin_state_up | True | | allowed_address_pairs | | | binding:host_id | nacx-kvm | | binding:profile | {} | | binding:vif_details | {"port_filter": true} | | binding:vif_type | bridge | | binding:vnic_type | normal | | device_id | | | device_owner | compute:Abiquo | | extra_dhcp_opts | | | fixed_ips | {"subnet_id": "c108f047-22f8-425c-9b33-305e74cc900e", "ip_address": "192.168.1.3"} | | id | 18365a93-63a5-4212-a23f-60adbff44024 | | mac_address | fa:16:3e:ff:e2:04 | | name | test-port | | network_id | 1a817eb7-ef09-455a-95dc-75e24d7b3d13 | | security_groups | 9105859e-1838-416a-87b7-33b764785d9e | | status | DOWN | | tenant_id | b7de212040bc4518b9fe11b53e8cbed7 | +-----------------------+------------------------------------------------------------------------------------+
21. Example: VM with a single IP and SSH access 20 July 2017 OpenStack BCN 7th birthday meetup 21
22. Example: VM with a single IP and SSH access 20 July 2017 OpenStack BCN 7th birthday meetup 22 Interfaces in the compute node [root@nacx-kvm ~]# ip link 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000 link/ether 52:54:00:1e:11:76 brd ff:ff:ff:ff:ff:ff 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000 link/ether 52:54:00:0c:55:0f brd ff:ff:ff:ff:ff:ff 4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT qlen 1000 link/ether 52:54:00:92:05:b4 brd ff:ff:ff:ff:ff:ff 5: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN mode DEFAULT qlen 1000 link/ether 52:54:00:92:05:b4 brd ff:ff:ff:ff:ff:ff Interfaces are managed by the compute agent (Nova), our custom hypervisor agent in this case Our agent needs to create the right one
23. Example: VM with a single IP and SSH access 20 July 2017 OpenStack BCN 7th birthday meetup 23 Link the port to an interface in the compute node +-----------------------+------------------------------------------------------------------------------------+ | Field | Value | +-----------------------+------------------------------------------------------------------------------------+ | admin_state_up | True | | allowed_address_pairs | | | binding:host_id | nacx-kvm | | binding:profile | {} | | binding:vif_details | {"port_filter": true} | | binding:vif_type | bridge | | binding:vnic_type | normal | | device_id | | | device_owner | compute:Abiquo | | extra_dhcp_opts | | | fixed_ips | {"subnet_id": "c108f047-22f8-425c-9b33-305e74cc900e", "ip_address": "192.168.1.3"} | | id | 18365a93-63a5-4212-a23f-60adbff44024 | | mac_address | fa:16:3e:ff:e2:04 | | name | test-port | | network_id | 1a817eb7-ef09-455a-95dc-75e24d7b3d13 | | security_groups | 9105859e-1838-416a-87b7-33b764785d9e | | status | DOWN | | tenant_id | b7de212040bc4518b9fe11b53e8cbed7 | +-----------------------+------------------------------------------------------------------------------------+
24. Example: VM with a single IP and SSH access 20 July 2017 OpenStack BCN 7th birthday meetup 24 Crea:ng the interface for the VM [root@nacx-kvm ~]# ip tuntap add name tap18365a93-63 mode tap [root@nacx-kvm ~]# ip link 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000 link/ether 52:54:00:1e:11:76 brd ff:ff:ff:ff:ff:ff 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000 link/ether 52:54:00:0c:55:0f brd ff:ff:ff:ff:ff:ff 4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT qlen 1000 link/ether 52:54:00:92:05:b4 brd ff:ff:ff:ff:ff:ff 5: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN mode DEFAULT qlen 1000 link/ether 52:54:00:92:05:b4 brd ff:ff:ff:ff:ff:ff 9: tap18365a93-63: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT qlen 1000 link/ether 4a:38:29:71:f4:8d brd ff:ff:ff:ff:ff:ff 10: eth1.78@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master brq1a817eb7-ef state UP mode DEFAULT qlen 1000 link/ether 52:54:00:0c:55:0f brd ff:ff:ff:ff:ff:ff 11: brq1a817eb7-ef: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT qlen 1000 link/ether 52:54:00:0c:55:0f brd ff:ff:ff:ff:ff:ff
25. Example: VM with a single IP and SSH access 20 July 2017 OpenStack BCN 7th birthday meetup 25 Crea:ng the interface for the VM [root@nacx-kvm ~]# ip tuntap add name tap18365a93-63 mode tap [root@nacx-kvm ~]# ip link 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000 link/ether 52:54:00:1e:11:76 brd ff:ff:ff:ff:ff:ff 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000 link/ether 52:54:00:0c:55:0f brd ff:ff:ff:ff:ff:ff 4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT qlen 1000 link/ether 52:54:00:92:05:b4 brd ff:ff:ff:ff:ff:ff 5: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN mode DEFAULT qlen 1000 link/ether 52:54:00:92:05:b4 brd ff:ff:ff:ff:ff:ff 9: tap18365a93-63: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT qlen 1000 link/ether 4a:38:29:71:f4:8d brd ff:ff:ff:ff:ff:ff 10: eth1.78@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master brq1a817eb7-ef state UP mode DEFAULT qlen 1000 link/ether 52:54:00:0c:55:0f brd ff:ff:ff:ff:ff:ff 11: brq1a817eb7-ef: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT qlen 1000 link/ether 52:54:00:0c:55:0f brd ff:ff:ff:ff:ff:ff
26. Example: VM with a single IP and SSH access 20 July 2017 OpenStack BCN 7th birthday meetup 26 And the port is finally ac:ve! (neutron) port-show test-port +-----------------------+------------------------------------------------------------------------------------+ | Field | Value | +-----------------------+------------------------------------------------------------------------------------+ | admin_state_up | True | | allowed_address_pairs | | | binding:host_id | nacx-kvm | | binding:profile | {} | | binding:vif_details | {"port_filter": true} | | binding:vif_type | bridge | | binding:vnic_type | normal | | device_id | | | device_owner | compute:Abiquo | | extra_dhcp_opts | | | fixed_ips | {"subnet_id": "c108f047-22f8-425c-9b33-305e74cc900e", "ip_address": "192.168.1.3"} | | id | 18365a93-63a5-4212-a23f-60adbff44024 | | mac_address | fa:16:3e:ff:e2:04 | | name | test-port | | network_id | 1a817eb7-ef09-455a-95dc-75e24d7b3d13 | | security_groups | 9105859e-1838-416a-87b7-33b764785d9e | | status | ACTIVE | | tenant_id | b7de212040bc4518b9fe11b53e8cbed7 | +-----------------------+------------------------------------------------------------------------------------+
27. Lessons learned 20 July 2017 OpenStack BCN 7th birthday meetup 27 •  Even though OpenStack is very modular, there are (hidden) hardcoded things that are important for integra:ons •  Open source is key when approaching new technology – Access to a community knowledge base of similar issues, use cases and experiences – Access to the source code is o-en the best way to understand technology
28. THANK YOU! 20 July 2017 OpenStack BCN 7th birthday meetup 28