• AIO: https://docs.openstack.org/openstack-ansible/queens/user/aio/quickstart.html
• Openstack network: https://docs.openstack.org/ocata/networking-guide/intro-basic-networking.html
• https://platform9.com/blog/install-openstack-using-openstack-ansible/
• https://github.com/nuagenetworks/nuage-openstack-ansible/wiki/Configure-OSA-Multi-node-Environment
• https://github.com/yan0s/ansible-openstack
• Sample: https://docs.openstack.org/openstack-ansible/latest/user/index.html
• https://elatov.github.io/2018/01/openstack-ansible-and-kolla-on-ubuntu-1604/
• Some arches:
  • http://bicofino.io/2017/09/05/openstack-ocata-deployment-with-openstack-ansible-and-ceph/
• AIO:
  • https://developer.rackspace.com/blog/life-without-devstack-openstack-development-with-osa/
• OVS network:
  • https://docs.openstack.org/openstack-ansible-os_neutron/latest/app-openvswitch.html
  • tutorial: https://medium.com/@travistruman/configuring-openstack-ansible-for-open-vswitch-b7e70e26009d
  • tutorial: http://trumant.github.io/openstack-ansible-openvswitch.html
  • https://elatov.github.io/2018/01/openstack-ansible-and-kolla-on-ubuntu-1604/
• HA Proxy
  • https://blog.envoyproxy.io/introduction-to-modern-network-load-balancing-and-proxying-a57f6ff80236
  • https://docs.oracle.com/cd/E90981_01/E90982/html/haproxy-keepalived.html
  • https://www.mirantis.com/blog/software-high-availability-load-balancing-openstack-cloud-api-servic/
• LXC
  • s
• nspawn
  • https://cloudnull.io/2017/06/nspawning-openstack-ansible/

Deploy Openstack on 3 VMs: 1 Controller + 1 Network + 1 Compute

Using kvm_quicklab project.

VM configuration:

  100 GB Disk
  4096 MB RAM
  2 CPU
  2 vNICs
  eth0 was on the "data" network
  eth1 was on the "system" network

<WRAP center round important 60%> OSA will overwrite system's ansible binary so better use a VM as deployment machine. </WRAP>

We use the controller node (Fedora 28 cloud) as deployment host.

  dnf upgrade (important)
  
  dnf install git ntp ntpdate openssh-server python-devel python-virtualenv btrfs-progs ansible iptables sudo '@Development Tools'
  systemctl start systemd-networkd.service
  systemctl enable systemd-networkd.service
  

Error: Unit dbus-org.freedesktop.resolve1.service not found

  cd /etc/systemd/system 
  sudo ln -s /usr/lib/systemd/system/systemd-resolved.service dbus-org.freedesktop.resolve1.service    

Error: python subprocess32 build

  dnf upgrade

• https://docs.openstack.org/project-deploy-guide/openstack-ansible/rocky/targethosts.html

VMs to be installed with OS components are connected as following:

[dang@gt130 os_dev_lab]$ brctl show
bridge name	bridge id		STP enabled	interfaces
br-2b718ca6738a		8000.024259ac89af	no		
br_ql_ext		8000.668a75761f2f	yes	compute-eth2
							controller-eth2
							network-eth2
br_ql_int		8000.66aca99d4771	yes	compute-eth1
							controller-eth1
							network-eth1
br_ql_mgmt		8000.72e36f4f30da	yes	compute-eth0
							controller-eth0
							network-eth0
docker0		8000.0242ebe08971	no		
virbr0		8000.5254008d5702	yes		virbr0-nic

IPs are configured statically

# ping all host in the subnet
nmap -sP 10.10.10.0/24

Starting Nmap 7.60 ( https://nmap.org ) at 2018-10-03 20:25 CEST
Nmap scan report for 10.10.10.1
Host is up (0.00042s latency).
Nmap scan report for os-compute (10.10.10.32)
Host is up (0.0017s latency).
Nmap scan report for os-network (10.10.10.84)
Host is up (0.00027s latency).
Nmap scan report for os-controller (10.10.10.190)
Host is up (0.00062s latency).
Nmap done: 256 IP addresses (4 hosts up) scanned in 3.73 seconds

• Tunneling technologies:
  • https://www.slideshare.net/enakai/how-vxlan-works-on-linux
• Ostack concepts: provider, self-service network:
  • https://docs.openstack.org/ocata/networking-guide/intro-os-networking.html
• Container network: https://docs.openstack.org/openstack-ansible/rocky/reference/architecture/container-networking.html
• Network: https://docs.openstack.org/ocata/networking-guide/deploy.html
  • bridge: https://docs.openstack.org/ocata/networking-guide/deploy-lb.html
  • ovs: https://docs.openstack.org/ocata/networking-guide/deploy-ovs.html
• Target hosts network layout: https://docs.openstack.org/ocata/install-guide-rdo/environment-networking.html

[clouduser@os-controller ~]$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br-mgmt state UP group default qlen 1000
    link/ether de:ad:be:80:7c:53 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::dcad:beff:fe80:7c53/64 scope link 
       valid_lft forever preferred_lft forever
3: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br-vlan state UP group default qlen 1000
    link/ether de:ad:be:80:7c:54 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::dcad:beff:fe80:7c54/64 scope link 
       valid_lft forever preferred_lft forever
4: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether de:ad:be:80:7c:55 brd ff:ff:ff:ff:ff:ff
    inet 10.30.30.192/24 brd 10.30.30.255 scope global dynamic ens5
       valid_lft 86279sec preferred_lft 86279sec
    inet6 fe80::dcad:beff:fe80:7c55/64 scope link 
       valid_lft forever preferred_lft forever
5: bond0: <BROADCAST,MULTICAST,MASTER> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 2e:f3:56:53:41:50 brd ff:ff:ff:ff:ff:ff
6: br-mgmt: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether de:ad:be:80:7c:53 brd ff:ff:ff:ff:ff:ff
    inet 10.10.10.10/24 brd 10.10.10.255 scope global br-mgmt
       valid_lft forever preferred_lft forever
    inet 10.10.10.9/24 brd 10.10.10.255 scope global secondary br-mgmt:0
       valid_lft forever preferred_lft forever
    inet6 fe80::dcad:beff:fe80:7c53/64 scope link 
       valid_lft forever preferred_lft forever
7: br-vlan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether de:ad:be:80:7c:54 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::dcad:beff:fe80:7c54/64 scope link 
       valid_lft forever preferred_lft forever
8: ens4.30@ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-vxlan state UP group default qlen 1000
    link/ether de:ad:be:80:7c:54 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::dcad:beff:fe80:7c54/64 scope link 
       valid_lft forever preferred_lft forever
9: br-vxlan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether de:ad:be:80:7c:54 brd ff:ff:ff:ff:ff:ff
    inet 10.40.40.10/24 brd 10.40.40.255 scope global br-vxlan
       valid_lft forever preferred_lft forever
    inet6 fe80::dcad:beff:fe80:7c54/64 scope link 
       valid_lft forever preferred_lft forever

[root@os-controller clouduser]# cat /etc/sysconfig/network-scripts/ifcfg-ens3
# internal network
DEVICE=ens3
TYPE=Ethernet
BOOTPROTO=none
ONBOOT=yes
BRIDGE=br-mgmt
[root@os-controller clouduser]# cat /etc/sysconfig/network-scripts/ifcfg-ens4
# internal network
DEVICE=ens4
TYPE=Ethernet
BOOTPROTO=no
ONBOOT=yes
BRIDGE=br-vlan
[root@os-controller clouduser]# cat /etc/sysconfig/network-scripts/ifcfg-ens4.30 
# internal network
DEVICE=ens4.30
BOOTPROTO=none
ONBOOT=yes
#IPADDR=192.168.1.1
#NETMASK=255.255.255.0
#NETWORK=192.168.1.0
USERCTL=no
VLAN=yes
BRIDGE=br-vxlan
[root@os-controller clouduser]# cat /etc/sysconfig/network-scripts/ifcfg-ens5
# external network
DEVICE=ens5
TYPE=Ethernet
BOOTPROTO=dhcp
ONBOOT=yes
[root@os-controller clouduser]# cat /etc/sysconfig/network-scripts/ifcfg-br-vlan 
DEVICE=br-vlan
TYPE=Bridge
ONBOOT=yes
BOOTPROTO=none
[root@os-controller clouduser]# cat /etc/sysconfig/network-scripts/ifcfg-br-vxlan 
DEVICE=br-vxlan
TYPE=Bridge
IPADDR=10.40.40.10
NETMASK=255.255.255.0
ONBOOT=yes
BOOTPROTO=none
[root@os-controller clouduser]# cat /etc/sysconfig/network-scripts/ifcfg-br-mgmt
DEVICE=br-mgmt
TYPE=Bridge
IPADDR=10.10.10.10
NETMASK=255.255.255.0
ONBOOT=yes
BOOTPROTO=none
[root@os-controller clouduser]# cat /etc/sysconfig/network-scripts/ifcfg-br-mgmt:0
DEVICE=br-mgmt:0
TYPE=Bridge
IPADDR=10.10.10.9
NETMASK=255.255.255.0
ONBOOT=yes
BOOTPROTO=none

these do not have ifcfg-br-mgmt:0

Configuration reflect the target host network setting and OpenStack architecture.

Autogenerate usersecrets.yml. Change keystoneauthadminpassword: admin

cidr_networks:

container: 10.10.10.0/24
tunnel: 10.40.40.0/24
#storage: 172.29.244.0/24

used_ips:

1. “10.10.10.1,10.10.10.50”
2. “10.40.40.1,10.40.40.50”

globaloverrides: # The internal and external VIP should be different IPs, however they # do not need to be on separate networks. #externallbvipaddress: 10.10.10.9

external_lb_vip_address: 10.10.10.10
internal_lb_vip_address: 10.10.10.10
management_bridge: "br-mgmt"
provider_networks:
  - network:
      container_bridge: "br-mgmt"
      container_type: "veth"
      container_interface: "eth1"
      ip_from_q: "container"
      type: "raw"
      group_binds:
        - all_containers
        - hosts
      is_container_address: true
  - network:
      container_bridge: "br-vxlan"
      container_type: "veth"
      # vir eth in lxc container or tap
      container_interface: "eth10"
      ip_from_q: "tunnel"
      type: "vxlan"
      range: "1:1000"
      net_name: "vxlan"
      group_binds:
        - neutron_linuxbridge_agent
  - network:
      container_bridge: "br-vlan"
      container_type: "veth"
      container_interface: "eth12"
      host_bind_override: "eth12"
      type: "flat"
      net_name: "flat"
      group_binds:
        - neutron_linuxbridge_agent
  - network:
      container_bridge: "br-vlan"
      container_type: "veth"
      container_interface: "eth11"
      type: "vlan"
      range: "101:200,301:400"
      net_name: "vlan"
      group_binds:
        - neutron_linuxbridge_agent
  #- network:
  #    container_bridge: "br-storage"
  #    container_type: "veth"
  #    container_interface: "eth2"
  #    ip_from_q: "storage"
  #    type: "raw"
  #    group_binds:
  #      - glance_api
  #      - cinder_api
  #      - cinder_volume
  #      - nova_compute

shared-infra_hosts:

infra1:
  ip: 10.10.10.10

repo-infra_hosts:

infra1:
  ip: 10.10.10.10

haproxy_hosts:

infra1:
  ip: 10.10.10.10

identity_hosts:

infra1:
  ip: 10.10.10.10

storage-infra_hosts:

infra1:
  ip: 10.10.10.10

image_hosts:

infra1:
  ip: 10.10.10.10

compute-infra_hosts:

infra1:
  ip: 10.10.10.10

orchestration_hosts:

infra1:
  ip: 10.10.10.10

dashboard_hosts:

infra1:
  ip: 10.10.10.10

network_hosts:

network:
  ip: 10.10.10.11

compute_hosts:

compute1:
  ip: 10.10.10.12

</code>

  cp env.d/aio_metal.yml.example env.d/aio_metal.yml

• Check selinux (no permission to start nginx on port). Can't disable cause it's required.
• This file is generated from user config. Remove previous configuration.

 rm /etc/openstack_deploy/openstack_inventory.json

manually move haproxy.* in /etc/ssl/private/ to /etc/pki/tls/private and remove the former dir.

TASK [os_keystone : Create system links] **************************************************************************************************************************************************************************
ok: [infra1] => (item={u'dest': u'/etc/ssl/certs', u'src': u'/etc/pki/tls/certs'})
failed: [infra1] (item={u'dest': u'/etc/ssl/private', u'src': u'/etc/pki/tls/private'}) => {"changed": false, "gid": 0, "group": "root", "item": {"dest": "/etc/ssl/private", "src": "/etc/pki/tls/private"}, "mode": "0755", "msg": "refusing to convert between directory and link for /etc/ssl/private", "owner": "root", "path": "/etc/ssl/private", "secontext": "unconfined_u:object_r:cert_t:s0", "size": 6, "state": "directory", "uid": 0}

#####changed: [infra1] => (item={u'dest': u'/etc/ssl/private', u'src': u'/etc/ssl/private'})
ok: [infra1] => (item={u'dest': u'/var/log/apache2', u'src': u'/var/log/httpd'})

• set nonlocal bind: https://serverfault.com/questions/377529/haproxy-cannot-bind-socket-for-proxy-on-a-remote-machine

• selinux must be enable!
• Check selinux /var/log/audit/autdit.log (search nginx, port 5000, etc)

grep 1539369704.507:11792 /var/log/audit/audit.log | audit2why

* enalble prot bind: https://www.itadminstrator.com/2017/07/nginx-no-permission-to-bind-port-9080.html

• selinux:
  • https://serverfault.com/questions/521078/how-can-i-query-for-all-selinux-rules-default-file-contexts-etc-affecting-a-type

[root@os-controller clouduser]# grep 1539380598.687:22934 /var/log/audit/audit.log | audit2allow -m nginx5001

module nginx5001 1.0;

require {
	type httpd_t;
	type commplex_link_port_t;
	class tcp_socket name_connect;
}

#============= httpd_t ==============

#!!!! This avc can be allowed using the boolean 'httpd_can_network_connect'
allow httpd_t commplex_link_port_t:tcp_socket name_connect;
[root@os-controller clouduser]# setsebool -P httpd_can_network_connect 1

  selinux error
  less /var/log/audit/audit.log
  
  
  # semanage port -l | grep 8181
  intermapper_port_t             tcp      8181
  
  seinfo -x --type=intermapper_port_t /etc/selinux/targeted/policy/policy.31
  
  # use -M to create module .pp, -m nginx.te to create te file.
  grep 1539370891.083:12008 /var/log/audit/audit.log | audit2allow -m nginx

  module nginx 1.0;

  require {
type httpd_t;
type intermapper_port_t;
class tcp_socket name_bind;
  }

  #============= httpd_t ==============
  allow httpd_t intermapper_port_t:tcp_socket name_bind;
  
  # TE file must be built
  # audit2allow(1) man page already describes it:

  # Building module policy manually

     # Compile the module
     $ checkmodule -M -m -o local.mod local.te

     # Create the package
     $ semodule_package -o local.pp -m local.mod

     # Load the module into the kernel
     $ semodule -i local.pp
  ######
  # add port to type
  semanage port -a -t http_port_t -p tcp 8181
  ValueError: Port tcp/8181 already defined
  # replace add -a with modify -m
  semanage port -m -t http_port_t -p tcp 8181

• no 7,8 : https://wiki.centos.org/HowTos/SELinux#head-faa96b3fdd922004cdb988c1989e56191c257c01

7. Creating Custom SELinux Policy Modules with audit2allow
Sometimes there are occasions when none of the above methods deal with a given situation and we need to extend the SELinux policy by creating a custom policy module to allow for a certain set of conditions. For example, consider the postgrey service add-on for an smtp mail server. Our smtp server needs to communicate with postgrey over a Unix socket and that is something the default SELinux policy for our smtp server does not allow. Consequently the service is blocked by SELinux. This is an issue that can not be fixed by changing or restoring file type security contexts and isn't something that has a boolean value we can toggle to allow. We could disable SELinux protection of the smtp server through a boolean, which would be better than disabling SELinux completely, but that is still far from ideal.

If we switch SELinux into Permissive mode and run our mail server for a set period of time, we can log SELinux issues whilst still permitting access (as mentioned in Gathering Audit Logs in Permissive Mode). Checking our logs, we see the following SELinux AVC messages:


type=AVC msg=audit(1218128130.653:334): avc:  denied  { connectto } for  pid=9111 comm="smtpd" path="/var/spool/postfix/postgrey/socket"
scontext=system_u:system_r:postfix_smtpd_t:s0 tcontext=system_u:system_r:initrc_t:s0 tclass=unix_stream_socket
type=AVC msg=audit(1218128130.653:334): avc:  denied  { write } for  pid=9111 comm="smtpd" name="socket" dev=sda6 ino=39977017
scontext=system_u:system_r:postfix_smtpd_t:s0 tcontext=system_u:object_r:postfix_spool_t:s0 tclass=sock_file 
Then we can use 'audit2allow' to generate a set of policy rules that would allow the required actions. We can generate a local postgrey Type Enforcement policy file (postgreylocal.te):


# grep smtpd_t /var/log/audit/audit.log | audit2allow -m postgreylocal > postgreylocal.te
# cat postgreylocal.te
module postgreylocal 1.0;
require {
        type postfix_smtpd_t;
        type postfix_spool_t;
        type initrc_t;
        class sock_file write;
        class unix_stream_socket connectto;
}
#============= postfix_smtpd_t ==============
allow postfix_smtpd_t initrc_t:unix_stream_socket connectto;
allow postfix_smtpd_t postfix_spool_t:sock_file write; 
Above we see that we can grep the audit.log file for issues relating to our smtp server and pipe those issues to audit2allow which generates a set of rules that it thinks would permit the actions currently denied by the SELinux policy. Reviewing these rules we see our smtp server wants to connect and write to a Unix socket which we see from out logs is the Unix socket that the postgrey service is listening on. As this seems perfectly reasonable, we can go ahead and use audit2allow to make a custom policy module to allow these actions:


# grep smtpd_t /var/log/audit/audit.log | audit2allow -M postgreylocal 
We then load our postgrey policy module using the 'semodule' command into the current SELinux policy:


semodule -i postgreylocal.pp 
which will add our postgrey policy module to /etc/selinux/targeted/modules/active/modules/postgreylocal.pp. We can check the policy module loaded correctly by listing loaded modules with 'semodule -l'.

We can then continue to monitor our SELinux log files to check that our custom policy module works and once we are satisfied we can re-enable SELinux Enforcing mode and again benefit from SELinux protection of our now fully functional smtp server.

7.1. Manually Customizing Policy Modules
Often audit2allow will automatically create a custom policy module that will resolve a particular issue, but there are times when it doesn't get it quite right and we may want to manually edit and compile the policy module. For example, consider the following AVC audit log:


Summary:
SELinux is preventing postdrop (postfix_postdrop_t) "getattr" to
/var/log/httpd/error_log (httpd_log_t).
Detailed Description:
SELinux denied access requested by postdrop. It is not expected that this access
is required by postdrop and this access may signal an intrusion attempt. It is
also possible that the specific version or configuration of the application is
causing it to require additional access.
Allowing Access:
Sometimes labeling problems can cause SELinux denials. You could try to restore
the default system file context for /var/log/httpd/error_log,
restorecon -v '/var/log/httpd/error_log'
If this does not work, there is currently no automatic way to allow this access.
Instead, you can generate a local policy module to allow this access - see FAQ
(http://fedora.redhat.com/docs/selinux-faq-fc5/#id2961385) Or you can disable
SELinux protection altogether. Disabling SELinux protection is not recommended.
Please file a bug report (http://bugzilla.redhat.com/bugzilla/enter_bug.cgi)
against this package.
Additional Information:
Source Context                system_u:system_r:postfix_postdrop_t
Target Context                root:object_r:httpd_log_t
Target Objects                /var/log/httpd/error_log [ file ]
Source                        postdrop
Source Path                   /usr/sbin/postdrop
Port                          <Unknown>
Host                          sanitized
Source RPM Packages           postfix-2.3.3-2
Target RPM Packages
Policy RPM                    selinux-policy-2.4.6-137.1.el5
Selinux Enabled               True
Policy Type                   targeted
MLS Enabled                   True
Enforcing Mode                Enforcing
Plugin Name                   catchall_file
Host Name                     sanitized
Platform                      Linux sanitized 2.6.18-53.1.21.el5 #1 SMP Tue
                              May 20 09:35:07 EDT 2008 x86_64 x86_64
Alert Count                   599
First Seen                    Wed Jul  2 08:27:15 2008
Last Seen                     Sun Aug 10 22:47:52 2008
Local ID                      c303a4ea-8e7a-4acc-9118-9cc61c6a2ec8
Line Numbers
Raw Audit Messages

host=sanitized type=AVC msg=audit(1218397672.372:352): avc:  denied  { getattr } for  pid=4262 comm="postdrop"
path="/var/log/httpd/error_log" dev=md2 ino=117005 scontext=system_u:system_r:postfix_postdrop_t:s0
tcontext=root:object_r:httpd_log_t:s0 tclass=file
host=sanitized type=SYSCALL msg=audit(1218397672.372:352): arch=c000003e syscall=5 success=no exit=-13 a0=2
a1=7fffd6febca0 a2=7fffd6febca0 a3=0 items=0 ppid=4261 pid=4262 auid=4294967295 uid=48 gid=48 euid=48 suid=48
fsuid=48 egid=90 sgid=90 fsgid=90 tty=(none) comm="postdrop" exe="/usr/sbin/postdrop"
subj=system_u:system_r:postfix_postdrop_t:s0 key=(null) 
Running audit2allow on the above error, and reviewing the resultant postfixlocal.te policy file we see:


    # grep postdrop /var/log/audit/audit.log | audit2allow -M postfixlocal
    # cat postfixlocal.te
    module postfixlocal 1.0;
    require {
            type httpd_log_t;
            type postfix_postdrop_t;
            class dir getattr;
            class file { read getattr };
    }
    #============= postfix_postdrop_t ==============
    allow postfix_postdrop_t httpd_log_t:file getattr; 

Hopefully the first thing to strike us here is why does postdrop needs access to /var/log/httpd/error_log? Presumably this isn't something we would expect so we have to assess if this is an action we wish to allow or not. We have a number of options. We could just ignore the error and allow SELinux to continue blocking and logging access attempts or we could allow the action by creating the custom policy module as suggested by audit2allow. Alternatively we can edit the custom policy module .te file to prevent auditing of this particular error whilst still allowing SELinux to continue preventing access. We do this by editing the allow line, changing it to dontaudit:


    #============= postfix_postdrop_t ==============
    dontaudit postfix_postdrop_t httpd_log_t:file getattr; 

Now we can manually compile and load the edited custom policy module:


    # checkmodule -M -m -o postfixlocal.mod postfixlocal.te
    # semodule_package -o postfixlocal.pp -m postfixlocal.mod
    # semodule -i postfixlocal.pp 

Access to /var/log/httpd/error_log by postdrop will still be prevented by SELinux but we won't receive constant alerts and error messages filling up our log files each time access is blocked.

• https://danwalsh.livejournal.com/24750.html

  • port: https://serverfault.com/questions/790404/selinux-error-valueerror-port-tcp-5000-already-defined