Commit Graph
174 Commits
Author SHA1 Message Date
Christian Iversen 8b07c88df8 [hermon] Add support for port management event
Inspired by Flexboot, the function hermon_event_port_mgmnt_change() is
added to handle the HERMON_EV_PORT_MGMNT_CHANGE event type, which
updates the Infiniband subsystem.

Signed-off-by: Christian Iversen <ci@iversenit.dk>
Modified-by: Michael Brown <mcb30@ipxe.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2021-02-01 11:44:54 +00:00
Christian Iversen d948ac6c61 [hermon] Adjust Ethernet work queue size
Hermon Ethernet work queues have more RX than TX entries, unlike most
other drivers.  This is possibly the source of some stochastic
deadlocks previously experienced with this driver.

Update the sizes to be in line with other drivers, and make them
slightly larger for better performance.  These new queue sizes have
been found to work well with ConnectX-3 hardware.

Signed-off-by: Christian Iversen <ci@iversenit.dk>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2021-02-01 11:12:26 +00:00
Michael Brown e62c3e3513 [hermon] Use reset value suitable for ConnectX-3
The programming documentation states that the reset magic value is
"0x00000001 (Big Endian)", and the current code matches this by using
the value 0x01000000 for the implicitly little-endian writel().

Inspection of the FlexBoot source code reveals an exciting variety of
reset values, some suggestive of confusion around endianness.

Experimentation suggests that the value 0x01000001 works reliably
across a wide range of hardware.

Debugged-by: Christian Iversen <ci@iversenit.dk>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2021-02-01 01:53:15 +00:00
Christian Iversen 2e3d5909ee [hermon] Clean up whitespace in hermon.c
Signed-off-by: Christian Iversen <ci@iversenit.dk>
2021-02-01 01:48:29 +00:00
Michael Brown def46cf344 [hermon] Limit link poll frequency in DOWN state
Some older versions of the hardware (and/or firmware) do not report an
event when an Infiniband link reaches the INIT state.  The driver
works around this missing event by calling ib_smc_update() on each
event queue poll while the link is in the DOWN state.

Commit 6cb12ee ("[hermon] Increase polling rate for command
completions") addressed this by speeding up the time taken to issue
each command invoked by ib_smc_update().  Experimentation shows that
the impact is still significant: for example, in a situation where an
unplugged port is opened, the throughput on the other port can be
reduced by over 99%.

Fix by throttling the rate at which link polling is attempted.

Debugged-by: Christian Iversen <ci@iversenit.dk>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2021-01-31 23:29:45 +00:00
Christian Iversen 43d72d0087 [hermon] Perform clean MPT unmap on device shutdown
This change is ported from Flexboot sources.  When stopping a Hermon
device, perform hermon_unmap_mpt() which runs HERMON_HCR_HW2SW_MPT to
bring the Memory Protection Table (MPT) back to software control.

Signed-off-by: Christian Iversen <ci@iversenit.dk>
Modified-by: Michael Brown <mcb30@ipxe.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2021-01-29 00:46:53 +00:00
Christian Iversen 699b9f1d1b [hermon] Use Ethernet MAC as eIPoIB local EMAC
The eIPoIB local Ethernet MAC is currently constructed from the port
GUID.  Given a base GUID/MAC value of N, Mellanox seems to populate:

  Node GUID:   N + 0
  Port 1 GUID: N + 1
  Port 2 GUID: N + 2

and

  Port 1 MAC:  N + 0
  Port 2 MAC:  N + 1

This causes a duplicate local MAC address when port 1 is configured as
Infiniband and port 2 as Ethernet, since both will derive their MAC
address as (N + 1).

Fix by using the port's Ethernet MAC as the eIPoIB local EMAC.  This
is a behavioural change that could potentially break configurations
that rely on the local EMAC value, such as a DHCP server relying on
the chaddr field for DHCP reservations.

Signed-off-by: Christian Iversen <ci@iversenit.dk>
Modified-by: Michael Brown <mcb30@ipxe.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2021-01-29 00:13:46 +00:00
Christian Iversen 6cb12ee2b0 [hermon] Increase polling rate for command completions
Some older versions of the hardware (and/or firmware) do not report an
event when an Infiniband link reaches the INIT state.  The driver
works around this missing event by calling ib_smc_update() on each
event queue poll while the link is in the DOWN state.  This results in
a very large number of commands being issued while any open Infiniband
link is in the DOWN state (e.g. unplugged), to the point that the 1ms
delay from waiting for each command to complete will noticeably affect
responsiveness.

Fix by decreasing the command completion polling delay from 1ms to
10us.

Signed-off-by: Christian Iversen <ci@iversenit.dk>
Modified-by: Michael Brown <mcb30@ipxe.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2021-01-28 23:47:00 +00:00
Michael Brown 7d32225b55 [hermon] Add event queue debug functions
Add hermon_dump_eqctx() for dumping the event queue context and
hermon_dump_eqes() for dumping any unconsumed event queue entries.

Originally-implemented-by: Christian Iversen <ci@iversenit.dk>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2021-01-28 22:30:56 +00:00
Christian Iversen 7c40227e18 [hermon] Increase command timeout from 2 to 10 seconds
Some commands (particularly in relation to device initialization) can
occasionally take longer than 2 seconds, and the Mellanox documentation
recommends a 10 second timeout.

Signed-off-by: Christian Iversen <ci@iversenit.dk>
2021-01-28 20:55:14 +00:00
Michael Brown cd126c41bb [hermon] Add assorted debug error messages
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2021-01-28 20:52:36 +00:00
Michael Brown ce45c8dc21 [hermon] Show "issuing command" messages only at DBGLVL_EXTRA
Originally-implemented-by: Christian Iversen <ci@iversenit.dk>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2021-01-28 17:29:36 +00:00
Christian Iversen a2893dc18a [hermon] Reorganize PCI ROM list and document well-known product names
Signed-off-by: Christian Iversen <ci@iversenit.dk>
2021-01-28 17:23:05 +00:00
Christian Iversen 0e788c8eda [golan] Backport typo fix in nodnic_prm.h: s/HERMON/NODNIC/
Signed-off-by: Christian Iversen <ci@iversenit.dk>
2021-01-28 17:19:22 +00:00
Christian Iversen 36a892a7c7 [arbel] Clean up whitespace in MT25218_PRM.h header
Signed-off-by: Christian Iversen <ci@iversenit.dk>
2021-01-28 17:14:08 +00:00
Christian Iversen 414c842f06 [hermon] Clean up whitespace in MT25408_PRM.h header
Signed-off-by: Christian Iversen <ci@iversenit.dk>
2021-01-28 17:10:47 +00:00
Christian Iversen b9de7e6eda [infiniband] Require drivers to specify the number of ports
Require drivers to report the total number of Infiniband ports.  This
is necessary to report the correct number of ports on devices with
dynamic port types.

For example, dual-port Mellanox cards configured for (eth, ib) would
be rejected by the subnet manager, because they report using "port 2,
out of 1".

Signed-off-by: Christian Iversen <ci@iversenit.dk>
2021-01-27 01:15:35 +00:00
Mohammed Taha ce841946df [golan] Add new PCI IDs
Signed-off-by: Mohammed <mohammedt@mellanox.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2020-12-28 13:55:30 +00:00
Michael Brown be1c87b722 [malloc] Rename malloc_dma() to malloc_phys()
The malloc_dma() function allocates memory with specified physical
alignment, and is typically (though not exclusively) used to allocate
memory for DMA.

Rename to malloc_phys() to more closely match the functionality, and
to create name space for functions that specifically allocate and map
DMA-capable buffers.

Signed-off-by: Michael Brown <mcb30@ipxe.org>
2020-11-05 19:13:52 +00:00
Michael Brown eecb75ba48 [pci] Update drivers to use pci_ioremap()
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2020-09-25 14:17:07 +01:00
Michael Brown 8a1d66c7ae [golan] Add explicit type casts for nodnic_queue_pair_type
GCC 10 emits warnings for implicit conversions of enumerated types.

The flexboot_nodnic code defines nodnic_queue_pair_type with values
identical to those of ib_queue_pair_type, and implicitly casts between
them.  Add an explicit cast to fix the warning.

Signed-off-by: Michael Brown <mcb30@ipxe.org>
2020-06-27 21:16:07 +01:00
Michael Brown 0b3000bbec [golan] Fix address-of-pointer bug for multicast attach/detach
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2019-08-17 17:51:18 +01:00
Mohammed 1cdf56f751 [golan] Add various new PCI device IDs
Signed-off-by: Mohammed <mohammedt@mellanox.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2019-05-02 11:00:18 +01:00
Michael Brown fe680c8228 [vlan] Provide vlan_netdev_rx() and vlan_netdev_rx_err()
The Hermon driver uses vlan_find() to identify the appropriate VLAN
device for packets that are received with the VLAN tag already
stripped out by the hardware.  Generalise this capability and expose
it for use by other network card drivers.

Signed-off-by: Michael Brown <mcb30@ipxe.org>
2019-04-27 20:25:00 +01:00
Ameer Mahagneh d5d4bf8870 [golan] Set log_max_qp to 1
This is required to work around a bug in some firmware versions.

Signed-off-by: Ameer Mahagneh <ameerm@mellanox.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2018-03-20 17:55:04 +02:00
Michael Brown 0778418e29 [golan] Do not assume all devices are identical
Remove the global variable shomron_nodnic_supported, since it may have
different values for different PCI devices.

Originally-fixed-by: Mohammed Taha <mohammedt@mellanox.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2018-03-18 15:41:21 +02:00
Michael Brown 5f85cbb9ee [build] Avoid implicit-fallthrough warnings on GCC 7
Reported-by: Vinson Lee <vlee@freedesktop.org>
Reported-by: Liang Yan <lyan@suse.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2017-03-29 10:36:45 +03:00
Raed Salem 1ff1eebcf7 [golan] Bug fixes and improved paging allocation method
Updates:
- revert Support for clear interrupt via BAR

Signed-off-by: Raed Salem <raeds@mellanox.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2017-03-23 16:03:40 +02:00
Michael Brown dea5b74475 [hermon] Assert that mapping length is non-zero
An (impossible) mapping length of zero produces a negative bit shift,
which is technically undefined.

Signed-off-by: Michael Brown <mcb30@ipxe.org>
2017-03-22 15:20:14 +02:00
Michael Brown 0ced99e97c [arbel] Assert that mapping length is non-zero
An (impossible) mapping length of zero produces a negative bit shift,
which is technically undefined.

Signed-off-by: Michael Brown <mcb30@ipxe.org>
2017-03-22 15:18:54 +02:00
Michael Brown 39ef530088 [infiniband] Return status code from ib_create_cq() and ib_create_qp()
Any underlying errors arising during ib_create_cq() or ib_create_qp()
are lost since the functions simply return NULL on error.  This makes
debugging harder, since a debug-enabled build is required to discover
the root cause of the error.

Fix by returning a status code from these functions, thereby allowing
any underlying errors to be propagated.

Signed-off-by: Michael Brown <mcb30@ipxe.org>
2017-03-22 11:18:02 +02:00
Michael Brown ae915aa5cc [qib7322] Use correct length for memset()
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2017-03-21 13:51:56 +02:00
Michael Brown 6ee15cbac3 [linda] Use correct length for memset()
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2017-03-21 13:50:51 +02:00
Michael Brown a5affc832e [arbel] Avoid potential integer overflow when calculating memory mappings
When the area to be mapped straddles the 2GB boundary, the expression
(high+size) will overflow on the first loop iteration.  Fix by using
(end-size), which cannot underflow.

Signed-off-by: Michael Brown <mcb30@ipxe.org>
2017-03-21 12:01:55 +02:00
Michael Brown 6ee62eb242 [hermon] Avoid potential integer overflow when calculating memory mappings
When the area to be mapped straddles the 2GB boundary, the expression
(high+size) will overflow on the first loop iteration.  Fix by using
(end-size), which cannot underflow.

Signed-off-by: Michael Brown <mcb30@ipxe.org>
2017-03-21 12:01:51 +02:00
Raed Salem 26050fd4c8 [golan] Update Connect-IB, ConnectX-4 and ConnectX-4 Lx (Infiniband) support
Updates:
- Nodnic: Support for arm cq doorbell via the UAR BAR
- Ensure hardware is quiescent when no interface is open - WinPE WA
- Support for clear interrupt via BAR
- Nodnic: Support for send TX doorbells via the UAR BAR
- Added ConnectX-5EX device
- Added ConnectX-5 device

Signed-off-by: Raed Salem <raeds@mellanox.com>
Modified-by: Michael Brown <mcb30@ipxe.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2016-12-08 09:35:52 +00:00
Michael Brown 08230599ef [golan] Fix building with GCC 6
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2016-05-04 16:01:32 +01:00
Wissam Shoukair 0eea8b5c3b [golan] Add missing iounmap()
Signed-off-by: Wissam Shoukair <wissams@mellanox.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2016-04-12 13:40:50 +01:00
Wissam Shoukair ffd959a1d6 [mlx_icmd] Fix compilation error in GCC versions newer than 4.6.4
Signed-off-by: Wissam Shoukair <wissams@mellanox.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2016-04-12 13:38:10 +01:00
Michael Brown 597521ef53 [qib7322] Validate payload length
There is no way for the hardware to give us an invalid length in the
LRH, since it must have parsed this length field in order to perform
header splitting.  However, this is difficult to prove conclusively.

Add an unnecessary length check to explicitly reject any packets
larger than the posted receive I/O buffer.

Signed-off-by: Michael Brown <mcb30@ipxe.org>
2016-03-30 07:31:51 +01:00
Michael Brown c9af896314 [linda] Validate payload length
There is no way for the hardware to give us an invalid length in the
LRH, since it must have parsed this length field in order to perform
header splitting.  However, this is difficult to prove conclusively.

Add an unnecessary length check to explicitly reject any packets
larger than the posted receive I/O buffer.

Signed-off-by: Michael Brown <mcb30@ipxe.org>
2016-03-30 07:27:09 +01:00
Christian Hesse 05027a7a12 [golan] Fix build error on some versions of gcc
Some versions of gcc complain that "'__bswap_variable_32' is static
but used in inline function 'golan_check_rc_and_cmd_status' which is
not static".

Fix by making golan_check_rc_and_cmd_status() a static inline.

Modified-by: Michael Brown <mcb30@ipxe.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2016-03-23 05:59:44 +00:00
Wissam Shoukair 0a20373a2f [golan] Add Connect-IB, ConnectX-4 and ConnectX-4 Lx (Infiniband) support
Signed-off-by: Wissam Shoukair <wissams@mellanox.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2016-03-22 17:55:55 +00:00
Michael Brown 860d5904fb [arbel] Fix received packet length
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2016-03-22 16:11:58 +00:00
Michael Brown 3ad028cf1c [hermon] Fix received packet length
Debugged-by: Wissam Shoukair <wissams@mellanox.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2016-03-22 16:09:18 +00:00
Michael Brown 57c63047e3 [arbel] Allocate space for GRH on UD queue pairs
As with the previous commit (for Hermon), allocate a separate ring
buffer to hold received GRHs.

Signed-off-by: Michael Brown <mcb30@ipxe.org>
2016-03-21 08:55:02 +00:00
Michael Brown e84c917f39 [hermon] Allocate space for GRH on UD queue pairs
The Infiniband specification (volume 1, section 11.4.1.2 "Post Receive
Request") notes that for UD QPs, the GRH will be placed in the first
40 bytes of the receive buffer if present.  (If no GRH is present,
which is normal, then the first 40 bytes of the receive buffer will be
unused.)

Mellanox hardware performs this placement automatically: other headers
will be stripped (and their values returned via the CQE), but the
first 40 bytes of the data buffer will be consumed by the (probably
non-existent) GRH.

This does not fit neatly into iPXE's internal abstraction, which
expects the data buffer to represent just the data payload with the
addresses from the GRH (if present) passed as additional parameters to
ib_complete_recv().

The end result of this discrepancy is that attempts to receive
full-sized 2048-byte IPoIB packets on Mellanox hardware will fail.

Fix by allocating a separate ring buffer to hold the received GRHs.

Signed-off-by: Michael Brown <mcb30@ipxe.org>
2016-03-21 08:18:15 +00:00
Michael Brown 4a861cc61c [qib7322] Add missing iounmap()
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2016-03-20 14:55:18 +00:00
Michael Brown bea9ee2397 [linda] Add missing iounmap()
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2016-03-20 14:54:08 +00:00
Michael Brown 692324905e [arbel] Add missing iounmap()
Signed-off-by: Michael Brown <mcb30@ipxe.org>
2016-03-20 14:52:01 +00:00