Felix configuration

A Felix configuration resource (FelixConfiguration) represents Felix configuration options for the cluster.

See Configuring Felix for more details.

Sample YAML

apiVersion: projectcalico.org/v3
kind: FelixConfiguration
metadata:
  name: default
spec:
  ipv6Support: false
  ipipMTU: 1400
  chainInsertMode: Append

Felix configuration definition

Metadata

Field	Description	Accepted Values	Schema
name	Unique name to describe this resource instance. Required.	Alphanumeric string with optional ., _, or -.	string

• Calico automatically creates a resource named default containing the global default configuration settings for Felix. You can use calicoctl to view and edit these settings
• The resources with the name node.<nodename> contain the node-specific overrides, and will be applied to the node <nodename>. When deleting a node the FelixConfiguration resource associated with the node will also be deleted.
• Resources with any other name can use the nodeSelector field to target a group of nodes by label (see Selector-scoped configuration below).

Selector-scoped configuration

note

This feature is a tech preview. The behavior described here may change in future releases.

FelixConfiguration resources whose name is not default and does not start with node. can include a nodeSelector field to restrict the configuration to nodes whose labels match the selector. This allows you to apply different Felix settings to groups of nodes without creating individual per-node resources.

Example

apiVersion: projectcalico.org/v3
kind: FelixConfiguration
metadata:
  name: gpu-nodes
spec:
  nodeSelector: "role == 'gpu'"
  logSeverityScreen: Debug
  bpfEnabled: true

This resource applies logSeverityScreen: Debug and bpfEnabled: true only to nodes with the label role=gpu.

Configuration precedence

When a selector-scoped FelixConfiguration matches a node, its settings are applied between per-node and global configuration in the priority order:

1. Environment variables (highest priority)
2. Felix configuration file
3. Host-specific FelixConfiguration resource (node.<nodename>)
4. Selector-scoped FelixConfiguration resources
5. Global FelixConfiguration resource (default)
6. Built-in defaults (lowest priority)

A selector-scoped setting overrides the same setting from the global default resource, but is itself overridden by a per-node node.<nodename> resource or environment variable.

Overlapping selectors

At most one selector-scoped FelixConfiguration should match any given node. If multiple selector-scoped resources match the same node, this is treated as a misconfiguration: the oldest resource by creation time is used and a warning is logged. This behavior may change in future releases.

Restrictions

• The nodeSelector field cannot be set on the default resource or on node.<nodename> resources. These names have fixed scoping semantics.
• The nodeSelector field uses the same selector syntax as other Calico resources.

Spec

Process: Feature detection/overrides

featureDetectOverride

Attribute	Value
Key	featureDetectOverride
Description	Used to override feature detection based on auto-detected platform capabilities. Values are specified in a comma separated list with no spaces, example; "SNATFullyRandom=true,MASQFullyRandom=false,RestoreSupportsLock=". A value of "true" or "false" will force enable/disable feature, empty or omitted values fall back to auto-detection.
Schema	String matching the regular expression ^([a-zA-Z0-9-_]+=(true|false|),)*([a-zA-Z0-9-_]+=(true|false|))?$.
Default	none

featureGates

Attribute	Value
Key	featureGates
Description	Used to enable or disable tech-preview Calico features. Values are specified in a comma separated list with no spaces, example; "BPFConnectTimeLoadBalancingWorkaround=enabled,XyZ=false". This is used to enable features that are not fully production ready.
Schema	String matching the regular expression ^([a-zA-Z0-9-_]+=([^=]+),)*([a-zA-Z0-9-_]+=([^=]+))?$.
Default	none

Process: Go runtime

goGCThreshold

Attribute	Value
Key	goGCThreshold
Description	Sets the Go runtime's garbage collection threshold. I.e. the percentage that the heap is allowed to grow before garbage collection is triggered. In general, doubling the value halves the CPU time spent doing GC, but it also doubles peak GC memory overhead. A special value of -1 can be used to disable GC entirely; this should only be used in conjunction with the GoMemoryLimitMB setting. This setting is overridden by the GOGC environment variable.
Schema	Integer: [-1,263-1]
Default	40

goMaxProcs

Attribute	Value
Key	goMaxProcs
Description	Sets the maximum number of CPUs that the Go runtime will use concurrently. A value of -1 means "use the system default"; typically the number of real CPUs on the system. this setting is overridden by the GOMAXPROCS environment variable.
Schema	Integer: [-1,263-1]
Default	-1

goMemoryLimitMB

Attribute	Value
Key	goMemoryLimitMB
Description	Sets a (soft) memory limit for the Go runtime in MB. The Go runtime will try to keep its memory usage under the limit by triggering GC as needed. To avoid thrashing, it will exceed the limit if GC starts to take more than 50% of the process's CPU time. A value of -1 disables the memory limit. Note that the memory limit, if used, must be considerably less than any hard resource limit set at the container or pod level. This is because felix is not the only process that must run in the container or pod. This setting is overridden by the GOMEMLIMIT environment variable.
Schema	Integer: [-1,263-1]
Default	-1

Process: Health port and timeouts

healthEnabled

Attribute	Value
Key	healthEnabled
Description	If set to true, enables Felix's health port, which provides readiness and liveness endpoints.
Schema	Boolean.
Default	false

healthHost

Attribute	Value
Key	healthHost
Description	The host that the health server should bind to.
Schema	String.
Default	localhost

healthPort

Attribute	Value
Key	healthPort
Description	The TCP port that the health server should bind to.
Schema	Integer: [0,65535]
Default	9099

healthTimeoutOverrides

Attribute	Value
Key	healthTimeoutOverrides
Description	Allows the internal watchdog timeouts of individual subcomponents to be overridden. This is useful for working around "false positive" liveness timeouts that can occur in particularly stressful workloads or if CPU is constrained. For a list of active subcomponents, see Felix's logs.
Schema	List of health timeout overrides: [{name: "<name>", timeout: "<duration>"}, ...] where <duration> is in the Go duration format, for example 1m30s.
Default	none

Process: Logging

logActionRateLimit

Attribute	Value
Key	logActionRateLimit
Description	Sets the rate of hitting a Log action. The value must be in the format "N/unit", where N is a number and unit is one of: second, minute, hour, or day. For example: "10/second" or "100/hour".
Schema	String matching the regular expression ^[1-9]\d{0,3}/(?:second|minute|hour|day)$.
Default	none

logActionRateLimitBurst

Attribute	Value
Key	logActionRateLimitBurst
Description	Sets the rate limit burst of hitting a Log action when LogActionRateLimit is enabled.
Schema	Integer: [0,263-1], [9999,263-1]
Default	5

logDebugFilenameRegex

Attribute	Value
Key	logDebugFilenameRegex
Description	Controls which source code files have their Debug log output included in the logs. Only logs from files with names that match the given regular expression are included. The filter only applies to Debug level logs.
Schema	String.
Default	none

logFilePath

Attribute	Value
Key	logFilePath
Description	The full path to the Felix log. Set to none to disable file logging.
Schema	String.
Default	/var/log/calico/felix.log

logPrefix

Attribute	Value
Key	logPrefix
Description	The log prefix that Felix uses when rendering LOG rules. It is possible to use the following specifiers to include extra information in the log prefix. - %t: Tier name. - %k: Kind (short names). - %n: Policy or profile name. - %p: Policy or profile name (namespace/name for namespaced kinds or just name for non namespaced kinds). Calico includes ": " characters at the end of the generated log prefix. Note that iptables shows up to 29 characters for the log prefix and nftables up to 127 characters. Extra characters are truncated.
Schema	String matching the regular expression ^([a-zA-Z0-9%: /_-])*$.
Default	calico-packet

logSeverityFile

Attribute	Value
Key	logSeverityFile
Description	The log severity above which logs are sent to the log file.
Schema	One of: Debug, Error, Fatal, Info, Trace, Warning.
Default	Info

logSeverityScreen

Attribute	Value
Key	logSeverityScreen
Description	The log severity above which logs are sent to the stdout.
Schema	One of: Debug, Error, Fatal, Info, Trace, Warning.
Default	Info

logSeveritySys

Attribute	Value
Key	logSeveritySys
Description	The log severity above which logs are sent to the syslog. Set to None for no logging to syslog.
Schema	One of: Debug, Error, Fatal, Info, Trace, Warning.
Default	Info

Process: Prometheus metrics

prometheusGoMetricsEnabled

Attribute	Value
Key	prometheusGoMetricsEnabled
Description	Disables Go runtime metrics collection, which the Prometheus client does by default, when set to false. This reduces the number of metrics reported, reducing Prometheus load.
Schema	Boolean.
Default	true

prometheusMetricsCAFile

Attribute	Value
Key	prometheusMetricsCAFile
Description	Defines the absolute path to the TLS CA certificate file used for securing the /metrics endpoint. This certificate must be valid and accessible by the calico-node process.
Schema	String.
Default	none

prometheusMetricsCertFile

Attribute	Value
Key	prometheusMetricsCertFile
Description	Defines the absolute path to the TLS certificate file used for securing the /metrics endpoint. This certificate must be valid and accessible by the calico-node process.
Schema	String.
Default	none

prometheusMetricsClientAuth

Attribute	Value
Key	prometheusMetricsClientAuth
Description	Specifies the client authentication type for the /metrics endpoint. This determines how the server validates client certificates. Default is "RequireAndVerifyClientCert".
Schema
Default	RequireAndVerifyClientCert

prometheusMetricsEnabled

Attribute	Value
Key	prometheusMetricsEnabled
Description	Enables the Prometheus metrics server in Felix if set to true.
Schema	Boolean.
Default	false

prometheusMetricsHost

Attribute	Value
Key	prometheusMetricsHost
Description	The host that the Prometheus metrics server should bind to.
Schema	String.
Default	none

prometheusMetricsKeyFile

Attribute	Value
Key	prometheusMetricsKeyFile
Description	Defines the absolute path to the private key file corresponding to the TLS certificate used for securing the /metrics endpoint. The private key must be valid and accessible by the calico-node process.
Schema	String.
Default	none

prometheusMetricsPort

Attribute	Value
Key	prometheusMetricsPort
Description	The TCP port that the Prometheus metrics server should bind to.
Schema	Integer: [0,65535]
Default	9091

prometheusProcessMetricsEnabled

Attribute	Value
Key	prometheusProcessMetricsEnabled
Description	Disables process metrics collection, which the Prometheus client does by default, when set to false. This reduces the number of metrics reported, reducing Prometheus load.
Schema	Boolean.
Default	true

prometheusWireGuardMetricsEnabled

Attribute	Value
Key	prometheusWireGuardMetricsEnabled
Description	Disables wireguard metrics collection, which the Prometheus client does by default, when set to false. This reduces the number of metrics reported, reducing Prometheus load.
Schema	Boolean.
Default	true

Data plane: Common

No matching group found for 'Data plane: Common'.

Data plane: iptables

No matching group found for 'Data plane: iptables'.

Data plane: nftables

No matching group found for 'Data plane: nftables'.

Data plane: eBPF

No matching group found for 'Data plane: eBPF'.

Data plane: Windows

No matching group found for 'Data plane: Windows'.

Data plane: OpenStack support

No matching group found for 'Data plane: OpenStack support'.

Data plane: XDP acceleration for iptables data plane

No matching group found for 'Data plane: XDP acceleration for iptables data plane'.

Overlay: VXLAN overlay

vxlanEnabled

Attribute	Value
Key	vxlanEnabled
Description	Overrides whether Felix should create the VXLAN tunnel device for IPv4 VXLAN networking. Optional as Felix determines this based on the existing IP pools.
Schema	Boolean.
Default	none

vxlanMTU

Attribute	Value
Key	vxlanMTU
Description	The MTU to set on the IPv4 VXLAN tunnel device. Optional as Felix auto-detects the MTU based on the MTU of the host's interfaces.
Schema	Integer
Default	0

vxlanMTUV6

Attribute	Value
Key	vxlanMTUV6
Description	The MTU to set on the IPv6 VXLAN tunnel device. Optional as Felix auto-detects the MTU based on the MTU of the host's interfaces.
Schema	Integer
Default	0

vxlanPort

Attribute	Value
Key	vxlanPort
Description	The UDP port number to use for VXLAN traffic.
Schema	Integer
Default	4789

vxlanVNI

Attribute	Value
Key	vxlanVNI
Description	The VXLAN VNI to use for VXLAN traffic. You may need to change this if the default value is in use on your system.
Schema	Integer
Default	4096

Overlay: IP-in-IP

ipipEnabled

Attribute	Value
Key	ipipEnabled
Description	Overrides whether Felix should configure an IPIP interface on the host. Optional as Felix determines this based on the existing IP pools.
Schema	Boolean.
Default	none

ipipMTU

Attribute	Value
Key	ipipMTU
Description	Controls the MTU to set on the IPIP tunnel device. Optional as Felix auto-detects the MTU based on the MTU of the host's interfaces.
Schema	Integer
Default	0

Overlay: WireGuard

No matching group found for 'Overlay: WireGuard'.

AWS integration

awsSrcDstCheck

Attribute	Value
Key	awsSrcDstCheck
Description	Controls whether Felix will try to change the "source/dest check" setting on the EC2 instance on which it is running. A value of "Disable" will try to disable the source/dest check. Disabling the check allows for sending workload traffic without encapsulation within the same AWS subnet.
Schema	One of: "Disable", "DoNothing", "Enable".
Default	DoNothing

Debug/test-only (generally unsupported)

debugDisableLogDropping

Attribute	Value
Key	debugDisableLogDropping
Description	Disables the dropping of log messages when the log buffer is full. This can significantly impact performance if log write-out is a bottleneck.
Schema	Boolean.
Default	false

debugHost

Attribute	Value
Key	debugHost
Description	The host IP or hostname to bind the debug port to. Only used if DebugPort is set.
Schema	String.
Default	localhost

debugMemoryProfilePath

Attribute	Value
Key	debugMemoryProfilePath
Description	The path to write the memory profile to when triggered by signal.
Schema	String.
Default	none

debugPort

Attribute	Value
Key	debugPort
Description	If set, enables Felix's debug HTTP port, which allows memory and CPU profiles to be retrieved. The debug port is not secure, it should not be exposed to the internet.
Schema	Integer: [0,65535]
Default	none

debugSimulateCalcGraphHangAfter

Attribute	Value
Key	debugSimulateCalcGraphHangAfter
Description	Used to simulate a hang in the calculation graph after the specified duration. This is useful in tests of the watchdog system only!
Schema	Duration string, for example 1m30s123ms or 1h5m.
Default	0s

debugSimulateDataplaneApplyDelay

Attribute	Value
Key	debugSimulateDataplaneApplyDelay
Description	Adds an artificial delay to every dataplane operation. This is useful for simulating a heavily loaded system for test purposes only.
Schema	Duration string, for example 1m30s123ms or 1h5m.
Default	0s

debugSimulateDataplaneHangAfter

Attribute	Value
Key	debugSimulateDataplaneHangAfter
Description	Used to simulate a hang in the dataplane after the specified duration. This is useful in tests of the watchdog system only!
Schema	Duration string, for example 1m30s123ms or 1h5m.
Default	0s

Usage reporting

usageReportingEnabled

Attribute	Value
Key	usageReportingEnabled
Description	Reports anonymous Calico version number and cluster size to projectcalico.org. Logs warnings returned by the usage server. For example, if a significant security vulnerability has been discovered in the version of Calico being used.
Schema	Boolean.
Default	true

usageReportingInitialDelay

Attribute	Value
Key	usageReportingInitialDelay
Description	Controls the minimum delay before Felix makes a report.
Schema	Duration string, for example 1m30s123ms or 1h5m.
Default	5m0s

usageReportingInterval

Attribute	Value
Key	usageReportingInterval
Description	Controls the interval at which Felix makes reports.
Schema	Duration string, for example 1m30s123ms or 1h5m.
Default	24h0m0s

Metadata

Field	Description	Accepted Values	Schema
name	Unique name to describe this resource instance. Required.	Alphanumeric string with optional ., _, or -.	string

• Calico automatically creates a resource named default containing the global default configuration settings for Felix. You can use calicoctl to view and edit these settings
• The resources with the name node.<nodename> contain the node-specific overrides, and will be applied to the node <nodename>. When deleting a node the FelixConfiguration resource associated with the node will also be deleted.

genericXDPEnabled and xdpRefreshInterval are only relevant when bpfEnabled is false and xdpEnabled is true; in other words when XDP is being used to accelerate denial-of-service prevention policies in the iptables data plane.

When bpfEnabled is true the "xdp" settings all have no effect; in BPF mode the implementation of policy is always accelerated, using the best available BPF technology.

Health Timeout Overrides

Felix has internal liveness and readiness watchdog timers that monitor its various loops. If a loop fails to "check in" within the allotted timeout then Felix will report non-Ready or non-Live on its health port (which is monitored by kubelet in a Kubernetes system). If Felix reports non-Live, this can result in the Pod being restarted.

In Kubernetes, if you see the calico-node Pod readiness or liveness checks fail intermittently, check the calico-node Pod log for a log from Felix that gives the overall health status (the list of components will depend on which features are enabled):

+---------------------------+---------+----------------+-----------------+--------+
|         COMPONENT         | TIMEOUT |    LIVENESS    |    READINESS    | DETAIL |
+---------------------------+---------+----------------+-----------------+--------+
| CalculationGraph          | 30s     | reporting live | reporting ready |        |
| FelixStartup              | 0s      | reporting live | reporting ready |        |
| InternalDataplaneMainLoop | 1m30s   | reporting live | reporting ready |        |
+---------------------------+---------+----------------+-----------------+--------+

If some health timeouts show as "timed out" it may help to apply an override using the healthTimeoutOverrides field:

...
spec:
  healthTimeoutOverrides:
  - name: InternalDataplaneMainLoop
    timeout: "5m"
  - name: CalculationGraph
    timeout: "1m30s"
  ...

A timeout value of 0 disables the timeout.

ProtoPort

Field	Description	Accepted Values	Schema
port	The exact port match	0-65535	int
protocol	The protocol match	tcp, udp, sctp	string
net	The CIDR match	any valid CIDR (e.g. 192.168.0.0/16)	string

Keep in mind that in the following example, net: "" and net: "0.0.0.0/0" are processed as the same in the policy enforcement.

  ...
spec:
  failsafeInboundHostPorts:
  - net: "192.168.1.1/32"
    port: 22
    protocol: tcp
  - net: ""
    port: 67
    protocol: udp
failsafeOutboundHostPorts:
  - net: "0.0.0.0/0"
    port: 67
    protocol: udp
  ...

RouteTableRange

The RouteTableRange option is now deprecated in favor of RouteTableRanges.

Field	Description	Accepted Values	Schema
min	Minimum index to use	1-250	int
max	Maximum index to use	1-250	int

RouteTableRanges

RouteTableRanges is a list of RouteTableRange objects:

Field	Description	Accepted Values	Schema
min	Minimum index to use	1 - 4294967295	int
max	Maximum index to use	1 - 4294967295	int

Each item in the RouteTableRanges list designates a range of routing tables available to Calico. By default, Calico will use a single range of 1-250. If a range spans Linux's reserved table range (253-255) then those tables are automatically excluded from the list. It's possible that other table ranges may also be reserved by third-party systems unknown to Calico. In that case, multiple ranges can be defined to target tables below and above the sensitive ranges:

 target tables 65-99, and 256-1000, skipping 100-255
calicoctl patch felixconfig default --type=merge -p '{"spec":{"routeTableRanges": [{"Min": 65, "Max": 99}, {"Min": 256, "Max": 1000}] }}

Note, for performance reasons, the maximum total number of routing tables that Felix will accept is 65535 (or 2*16).

Specifying both the RouteTableRange and RouteTableRanges arguments is not supported and will result in an error from the api.

AWS IAM Role/Policy for source-destination-check configuration

Setting awsSrcDstCheck to Disable will automatically disable source-destination-check on EC2 instances in a cluster, provided necessary IAM roles and policies are set. One of the policies assigned to IAM role of cluster nodes must contain a statement similar to the following:

{
    "Effect": "Allow",
        "Action": [
            "ec2:DescribeInstances",
            "ec2:ModifyNetworkInterfaceAttribute"
        ],
    "Resource": "*"
}

If there are no policies attached to node roles containing the above statement, attach a new policy. For example, if a node role is test-cluster-nodeinstance-role, click on the IAM role in AWS console. In the Permission policies list, add a new inline policy with the above statement to the new policy JSON definition. For detailed information, see AWS documentation.

For an EKS cluster, the necessary IAM role and policy is available by default. No further actions are needed.