Commit Graph

57 Commits

Author SHA1 Message Date
Dave Jiang
5e34b49756 dmaengine: idxd: add missing callback function to support DMA_INTERRUPT
commit 2112b8f4fb5cc35d1c384324763765953186b81f upstream.

When setting DMA_INTERRUPT capability, a callback function
dma->device_prep_dma_interrupt() is needed to support this capability.
Without setting the callback, dma_async_device_register() will fail dma
capability check.

Fixes: 4e5a4eb20393 ("dmaengine: idxd: set DMA_INTERRUPT cap bit")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/165101232637.3951447.15765792791591763119.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2022-06-14 18:32:47 +02:00
Dave Jiang
76b226eaf0 dmaengine: idxd: set DMA_INTERRUPT cap bit
[ Upstream commit 4e5a4eb20393b851590b4465f1197a8041c2076b ]

Even though idxd driver has always supported interrupt, it never actually
set the DMA_INTERRUPT cap bit. Rectify this mistake so the interrupt
capability is advertised.

Reported-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/164971497859.2201379.17925303210723708961.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2022-06-14 18:32:37 +02:00
Christophe JAILLET
6073af7815 dmaengine: idxd: Fix the error handling path in idxd_cdev_register()
[ Upstream commit aab08c1aac01097815fbcf10fce7021d2396a31f ]

If a call to alloc_chrdev_region() fails, the already allocated resources
are leaking.

Add the needed error handling path to fix the leak.

Fixes: 42d279f913 ("dmaengine: idxd: add char driver to expose submission portal to userland")
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/1b5033dcc87b5f2a953c413f0306e883e6114542.1650521591.git.christophe.jaillet@wanadoo.fr
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2022-06-09 10:21:19 +02:00
Dave Jiang
520aab8b72 dmaengine: idxd: add RO check for wq max_transfer_size write
[ Upstream commit 505a2d1032ae656b0a8c736be110255503941cde ]

Block wq_max_transfer_size_store() when the device is configured as
read-only and not configurable.

Fixes: d7aad5550e ("dmaengine: idxd: add support for configurable max wq xfer size")
Reported-by: Bernice Zhang <bernice.zhang@intel.com>
Tested-by: Bernice Zhang <bernice.zhang@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/164971488154.2200913.10706665404118545941.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2022-04-27 13:53:51 +02:00
Dave Jiang
9a3c026dc3 dmaengine: idxd: add RO check for wq max_batch_size write
[ Upstream commit 66903461ffed0b66fc3e0200082d4e09365aacdc ]

Block wq_max_batch_size_store() when the device is configured as read-only
and not configurable.

Fixes: e7184b159d ("dmaengine: idxd: add support for configurable max wq batch size")
Reported-by: Bernice Zhang <bernice.zhang@intel.com>
Tested-by: Bernice Zhang <bernice.zhang@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/164971493551.2201159.1942042593642155209.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2022-04-27 13:53:51 +02:00
Dave Jiang
48271d10bf dmaengine: idxd: fix wq slot allocation index check
[ Upstream commit 673d812d30be67942762bb9e8548abb26a3ba4a7 ]

The sbitmap wait and allocate routine checks the index that is returned
from sbitmap_queue_get(). It should be idxd >= 0 as 0 is also a valid
index. This fixes issue where submission path hangs when WQ size is 1.

Fixes: 0705107fcc ("dmaengine: idxd: move submission to sbitmap_queue")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162697645067.3478714.506720687816951762.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-09-26 14:08:58 +02:00
Dave Jiang
4d74c98023 dmaengine: idxd: add missing dsa driver unregister
[ Upstream commit 077cdb355b3d8ee0f258856962e6dac06e744401 ]

The idxd_unregister_driver() has never been called for the idxd driver upon
removal. Add fix to call unregister driver on module removal.

Fixes: c52ca47823 ("dmaengine: idxd: add configuration component of driver")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/161947994449.1053102.13189942817915448216.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-06-23 14:42:40 +02:00
Dave Jiang
dd41a0e515 dmaengine: idxd: fix cdev setup and free device lifetime issues
[ Upstream commit 04922b7445a1950b86f130a1fe8c52cc27b3e30b ]

The char device setup and cleanup has device lifetime issues regarding when
parts are initialized and cleaned up. The initialization of struct device is
done incorrectly. device_initialize() needs to be called on the 'struct
device' and then additional changes can be added. The ->release() function
needs to be setup via device_type before dev_set_name() to allow proper
cleanup. The change re-parents the cdev under the wq->conf_dev to get
natural reference inheritance. No known dependency on the old device path exists.

Reported-by: Jason Gunthorpe <jgg@nvidia.com>
Fixes: 42d279f913 ("dmaengine: idxd: add char driver to expose submission portal to userland")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Link: https://lore.kernel.org/r/161852987721.2203940.1478218825576630810.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-05-19 10:13:04 +02:00
Dave Jiang
975c4b2b99 dmaengine: idxd: fix dma device lifetime
[ Upstream commit 397862855619271296e46d10f7dfa7bafe71eb81 ]

The devm managed lifetime is incompatible with 'struct device' objects that
resides in idxd context. This is one of the series that clean up the idxd
driver 'struct device' lifetime. Remove embedding of dma_device and dma_chan
in idxd since it's not the only interface that idxd will use. The freeing of
the dma_device will be managed by the ->release() function.

Reported-by: Jason Gunthorpe <jgg@nvidia.com>
Fixes: bfe1d56091 ("dmaengine: idxd: Init and probe for Intel data accelerators")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Link: https://lore.kernel.org/r/161852983001.2203940.14817017492384561719.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-05-19 10:13:04 +02:00
Colin Ian King
5756f757c7 dmaengine: idxd: Fix potential null dereference on pointer status
[ Upstream commit 28ac8e03c43dfc6a703aa420d18222540b801120 ]

There are calls to idxd_cmd_exec that pass a null status pointer however
a recent commit has added an assignment to *status that can end up
with a null pointer dereference.  The function expects a null status
pointer sometimes as there is a later assignment to *status where
status is first null checked.  Fix the issue by null checking status
before making the assignment.

Addresses-Coverity: ("Explicit null dereferenced")
Fixes: 89e3becd8f82 ("dmaengine: idxd: check device state before issue command")
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/20210415110654.1941580-1-colin.king@canonical.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-05-19 10:13:03 +02:00
Dave Jiang
e5eb9757fe dmaengine: idxd: fix wq cleanup of WQCFG registers
[ Upstream commit ea9aadc06a9f10ad20a90edc0a484f1147d88a7a ]

A pre-release silicon erratum workaround where wq reset does not clear
WQCFG registers was leaked into upstream code. Use wq reset command
instead of blasting the MMIO region. This also address an issue where
we clobber registers in future devices.

Fixes: da32b28c95 ("dmaengine: idxd: cleanup workqueue config after disabling")
Reported-by: Shreenivaas Devarajan <shreenivaas.devarajan@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/161824330020.881560.16375921906426627033.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-04-21 13:00:51 +02:00
Dave Jiang
4ecf255952 dmaengine: idxd: fix wq size store permission state
[ Upstream commit 0fff71c5a311e1264988179f7dcc217fda15fadd ]

WQ size can only be changed when the device is disabled. Current code
allows change when device is enabled but wq is disabled. Change the check
to detect device state.

Fixes: c52ca47823 ("dmaengine: idxd: add configuration component of driver")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/161782558755.107710.18138252584838406025.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-04-21 13:00:50 +02:00
Dave Jiang
db23b7b5ca dmaengine: idxd: fix opcap sysfs attribute output
[ Upstream commit ea6a5735d2a61b938a302eb3629272342a9e7c46 ]

The operation capability register is 256bits. The current output only
prints out the first 64bits. Fix to output the entire 256bits. The current
code omits operation caps from IAX devices.

Fixes: c52ca47823 ("dmaengine: idxd: add configuration component of driver")
Reported-by: Lucas Van <lucas.van@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/161645624963.2003736.829798666998490151.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-04-21 13:00:50 +02:00
Dave Jiang
a5ad12d5d6 dmaengine: idxd: Fix clobbering of SWERR overflow bit on writeback
[ Upstream commit ea941ac294d75d0ace50797aebf0056f6f8f7a7f ]

Current code blindly writes over the SWERR and the OVERFLOW bits. Write
back the bits actually read instead so the driver avoids clobbering the
OVERFLOW bit that comes after the register is read.

Fixes: bfe1d56091 ("dmaengine: idxd: Init and probe for Intel data accelerators")
Reported-by: Sanjay Kumar <sanjay.k.kumar@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/161352082229.3511254.1002151220537623503.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-04-21 13:00:50 +02:00
Dave Jiang
4f24543828 dmaengine: idxd: set DMA channel to be private
[ Upstream commit c06e424be5f5184468c5f761c0d2cf1ed0a4e0fc ]

Add DMA_PRIVATE attribute flag to idxd DMA channels. The dedicated WQs are
expected to be used by a single client and not shared. While doing NTB
testing this mistake was discovered, which prevented ntb_transport from
requesting DSA wqs as DMA channels via dma_request_channel().

Reported-by: Srinijia Kambham <srinija.kambham@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Tested-by: Srinijia Kambham <srinija.kambham@intel.com>
Fixes: 8f47d1a5e5 ("dmaengine: idxd: connect idxd to dmaengine subsystem")
Link: https://lore.kernel.org/r/161074758743.2184057.3388557138816350980.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-03-04 11:37:57 +01:00
Dave Jiang
40e3b5c128 dmaengine: idxd: check device state before issue command
[ Upstream commit 89e3becd8f821e507052e012d2559dcda59f538e ]

Add device state check before executing command. Without the check the
command can be issued while device is in halt state and causes the driver to
block while waiting for the completion of the command.

Reported-by: Sanjay Kumar <sanjay.k.kumar@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Tested-by: Sanjay Kumar <sanjay.k.kumar@intel.com>
Fixes: 0d5c10b4c8 ("dmaengine: idxd: add work queue drain support")
Link: https://lore.kernel.org/r/161219313921.2976211.12222625226450097465.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-02-17 11:02:25 +01:00
Dave Jiang
c39cdf559d dmaengine: idxd: fix misc interrupt completion
[ Upstream commit f5cc9ace24fbdf41b4814effbb2f9bad7046e988 ]

Nikhil reported the misc interrupt handler can sometimes miss handling
the command interrupt when an error interrupt happens near the same time.
Have the irq handling thread continue to process the misc interrupts until
all interrupts are processed. This is a low usage interrupt and is not
expected to handle high volume traffic. Therefore there is no concern of
this thread running for a long time.

Fixes: 0d5c10b4c8 ("dmaengine: idxd: add work queue drain support")
Reported-by: Nikhil Rao <nikhil.rao@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/161074755329.2183844.13295528344116907983.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2021-02-17 11:02:25 +01:00
Dave Jiang
285b57595d dmaengine: move channel device_node deletion to driver
commit e594443196d6e0ef3d3b30320c49b3a4d4f9a547 upstream.

Channel device_node deletion is managed by the device driver rather than
the dmaengine core. The deletion was accidentally introduced when making
channel unregister dynamic. It causes xilinx_dma module to crash on unload
as reported by Radhey. Remove chan->device_node delete in dmaengine and
also fix up idxd driver.

[   42.142705] Internal error: Oops: 96000044 [#1] SMP
[   42.147566] Modules linked in: xilinx_dma(-) clk_xlnx_clock_wizard uio_pdrv_genirq
[   42.155139] CPU: 1 PID: 2075 Comm: rmmod Not tainted 5.10.1-00026-g3a2e6dd7a05-dirty #192
[   42.163302] Hardware name: Enclustra XU5 SOM (DT)
[   42.167992] pstate: 40000005 (nZcv daif -PAN -UAO -TCO BTYPE=--)
[   42.173996] pc : xilinx_dma_chan_remove+0x74/0xa0 [xilinx_dma]
[   42.179815] lr : xilinx_dma_chan_remove+0x70/0xa0 [xilinx_dma]
[   42.185636] sp : ffffffc01112bca0
[   42.188935] x29: ffffffc01112bca0 x28: ffffff80402ea640

xilinx_dma_chan_remove+0x74/0xa0:
__list_del at ./include/linux/list.h:112 (inlined by)
__list_del_entry at./include/linux/list.h:135 (inlined by)
list_del at ./include/linux/list.h:146 (inlined by)
xilinx_dma_chan_remove at drivers/dma/xilinx/xilinx_dma.c:2546

Fixes: e81274cd6b ("dmaengine: add support to dynamic register/unregister of channels")
Reported-by: Radhey Shyam Pandey <radheys@xilinx.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Tested-by: Radhey Shyam Pandey <radhey.shyam.pandey@xilinx.com>
Link: https://lore.kernel.org/r/161099092469.2495902.5064826526660062342.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Cc: stable@vger.kernel.org # 5.9+
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-02-17 11:02:21 +01:00
Dan Carpenter
6e3c67976e dmaengine: idxd: off by one in cleanup code
commit ff58f7dd0c1352a01de3a40327895bd51e03de3a upstream.

The clean up is off by one so this will start at "i" and it should start
with "i - 1" and then it doesn't unregister the zeroeth elements in the
array.

Fixes: c52ca47823 ("dmaengine: idxd: add configuration component of driver")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/X9nFeojulsNqUSnG@mwanda
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2021-01-12 20:18:26 +01:00
Dave Jiang
8326be9f1c dmaengine: idxd: fix mapping of portal size
Portal size is 4k. Current code is mapping all 4 portals in a single chunk.
Restrict the mapped portal size to a single portal to ensure that submission
only goes to the intended portal address.

Fixes: c52ca47823 ("dmaengine: idxd: add configuration component of driver")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/160513342642.510187.16450549281618747065.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-11-16 22:38:20 +05:30
Dave Jiang
484f910e93 dmaengine: idxd: fix wq config registers offset programming
DSA spec v1.1 [1] updated to include a stride size register for WQ
configuration that will specify how much space is reserved for the WQ
configuration register set. This change is expected to be in the final
gen1 DSA hardware. Fix the driver to use WQCFG_OFFSET() for all WQ
offset calculation and fixup WQCFG_OFFSET() to use the new calculated
wq size.

[1]: https://software.intel.com/content/www/us/en/develop/download/intel-data-streaming-accelerator-preliminary-architecture-specification.html

Fixes: bfe1d56091 ("dmaengine: idxd: Init and probe for Intel data accelerators")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/160383444959.48058.14249265538404901781.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-10-28 11:02:41 +05:30
Vinod Koul
4c80e93239 Linux 5.9-rc4
-----BEGIN PGP SIGNATURE-----
 
 iQFSBAABCAA8FiEEq68RxlopcLEwq+PEeb4+QwBBGIYFAl9VerweHHRvcnZhbGRz
 QGxpbnV4LWZvdW5kYXRpb24ub3JnAAoJEHm+PkMAQRiGhc4H/iHD6qLdB36gZB6K
 oc2nJyrqyWitv4ti2Mnt5PA7o4wX4l6nnr1QvoaJ4BRs5Ja1czRvb2XDmdzqAoIA
 xITGoafqaAeDfxQ91bWrJsVN0pCRKiGwddXlU7TWmqw/riAkfOqi6GYKvav4biJH
 +n1mUPQb1M2IbRFsqkAS+ebKHq3CWaRvzKOEneS88nGlL5u31S9NAru8Ru/fkxRn
 6CwGcs1XRaBPYaZAhdfIb0NuatUlpkhPC9yhNS9up6SqrWmK3m65vmFVng6H0eCF
 fwn1jVztboY/XcNAi5sM9ExpQCql6WLQEEktVikqRDojC8fVtSx6W55tPt7qeaoO
 Z6t4/DA=
 =bcA4
 -----END PGP SIGNATURE-----

Merge tag 'v5.9-rc4' into next

Linux 5.9-rc4
2020-09-11 17:45:36 +05:30
Dave Jiang
ff18de55a6 dmaengine: idxd: add command status to idxd sysfs attribute
Export admin command status to sysfs attribute in order to allow user to
retrieve configuration error. Allows user tooling to retrieve the command
error and provide more user friendly error messages.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/159865278770.29455.8026892329182750127.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-09-03 12:40:12 +05:30
Dave Jiang
e7184b159d dmaengine: idxd: add support for configurable max wq batch size
Add sysfs attribute max_batch_size to wq in order to allow the max batch
size configured on a per wq basis. Add support code to configure
the valid user input on wq enable. This is a performance tuning
parameter.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/159865273617.29141.4383066301730821749.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-09-03 12:40:12 +05:30
Dave Jiang
d7aad5550e dmaengine: idxd: add support for configurable max wq xfer size
Add sysfs attribute max_xfer_size to wq in order to allow the max xfer
size configured on a per wq basis. Add support code to configure
the valid user input on wq enable. This is a performance tuning
parameter.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/159865265404.29141.3049399618578194052.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-09-03 12:40:12 +05:30
Dave Jiang
0ec083e50c dmaengine: idxd: clear misc interrupt cause after read
Move the clearing of misc interrupt cause to immediately after reading the
register in order to allow the next interrupt to be asserted.

Suggested-by: Nikhil Rao <nikhil.rao@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/159603824665.28647.5344356370364397996.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-08-17 10:49:11 +05:30
Dave Jiang
df841b17e8 dmaengine: idxd: reset states after device disable or reset
The state for WQs should be reset to disabled when a device is reset or
disabled.

Fixes: da32b28c95 ("dmaengine: idxd: cleanup workqueue config after disabling")
Reported-by: Mona Hossain <mona.hossain@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/159586777684.27150.17589406415773568534.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-08-17 10:25:04 +05:30
Vinod Koul
0b5ad7b952 Merge branch 'for-linus' into fixes
Signed-off-by: Vinod Koul <vkoul@kernel.org>

 Conflicts:
	drivers/dma/idxd/sysfs.c
2020-08-05 19:02:07 +05:30
Dave Jiang
4548a6ad3d dmaengine: idxd: move idxd interrupt handling to mask instead of ignore
Switch driver to use MSIX mask and unmask instead of the ignore bit.
When ignore bit is cleared, we must issue an MMIO read to ensure writes
have all arrived and check and process any additional completions. The
ignore bit does not queue up any pending MSIX interrupts. The mask bit
however does. Use API call from interrupt subsystem to mask MSIX
interrupt since the hardware does not have convenient mask bit register.

Suggested-by: Ashok Raj <ashok.raj@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/159319517621.70410.11816465052708900506.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-07-13 14:48:02 +05:30
Dave Jiang
0d5c10b4c8 dmaengine: idxd: add work queue drain support
Add wq drain support. When a wq is being released, it needs to wait for
all in-flight operation to complete.  A device control function
idxd_wq_drain() has been added to facilitate this. A wq drain call
is added to the char dev on release to make sure all user operations are
complete. A wq drain is also added before the wq is being disabled.

A drain command can take an unpredictable period of time. Interrupt support
for device commands is added to allow waiting on the command to
finish. If a previous command is in progress, the new submitter can block
until the current command is finished before proceeding. The interrupt
based submission will submit the command and then wait until a command
completion interrupt happens to complete. All commands are moved to the
interrupt based command submission except for the device reset during
probe, which will be polled.

Fixes: 42d279f913 ("dmaengine: idxd: add char driver to expose submission portal to userland")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Tony Luck <tony.luck@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Link: https://lore.kernel.org/r/159319502515.69593.13451647706946040301.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-07-13 14:48:02 +05:30
Dave Jiang
e3122822a7 dmaengine: idxd: fix misc interrupt handler thread unmasking
Fix unmasking of misc interrupt handler when completing normal. It exits
early and skips the unmasking with the current implementation. Fix to
unmask interrupt when exiting normally.

Fixes: bfe1d56091 ("dmaengine: idxd: Init and probe for Intel data accelerators")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/159311256528.855.11527922406329728512.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-07-02 19:02:20 +05:30
Dave Jiang
da32b28c95 dmaengine: idxd: cleanup workqueue config after disabling
After disabling a device, we should clean up the internal state for
the wqs and zero out the configuration registers. Without doing so can cause
issues when the user reprogram the wqs.

Fixes: c52ca47823 ("dmaengine: idxd: add configuration component of driver")
Reported-by: Yixin Zhang <yixin.zhang@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Tested-by: Yixin Zhang <yixin.zhang@intel.com>
Link: https://lore.kernel.org/r/159311264246.1198.11955791213681679428.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-07-02 19:02:19 +05:30
Dave Jiang
0705107fcc dmaengine: idxd: move submission to sbitmap_queue
Kill the percpu-rwsem for work submission in favor of an sbitmap_queue.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Tony Luck <tony.luck@intel.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Link: https://lore.kernel.org/r/159225446631.68253.8860709181621260997.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-06-24 12:55:09 +05:30
Nikhil Rao
66983bc18f dmaengine: idxd: fix cdev locking for open and release
add the wq lock in cdev open and release call. This fixes
race conditions observed in the open and close routines.

Fixes: 42d279f913 ("dmaengine: idxd: add char driver to expose submission portal to userland")
Signed-off-by: Nikhil Rao <nikhil.rao@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/159285824892.64944.2905413694915141834.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-06-24 11:17:46 +05:30
Dave Jiang
47ec7f09bc dmaengine: cookie bypass for out of order completion
The cookie tracking in dmaengine expects all submissions completed in
order. Some DMA devices like Intel DSA can complete submissions out of
order, especially if configured with a work queue sharing multiple DMA
engines. Add a status DMA_OUT_OF_ORDER that tx_status can be returned for
those DMA devices. The user should use callbacks to track the completion
rather than the DMA cookie. This would address the issue of dmatest
complaining that descriptors are "busy" when the cookie count goes
backwards due to out of order completion. Add DMA_COMPLETION_NO_ORDER
DMA capability to allow the driver to flag the device's ability to complete
operations out of order.

Reported-by: Swathi Kovvuri <swathi.kovvuri@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Tested-by: Swathi Kovvuri <swathi.kovvuri@intel.com>
Link: https://lore.kernel.org/r/158939557151.20335.12404113976045569870.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-06-17 19:45:14 +05:30
Linus Torvalds
c90e7945e3 dmaengine updates for v5.8-rc1
Bunch of updates to drivers like dmatest, dw-edma, ioat,
 mmp-tdma and k3-udma along with Renesas binding update to json-schema
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+vs47OPLdNbVcHzyfBQHDyUjg0cFAl7fdVEACgkQfBQHDyUj
 g0e7PBAAhtKUb1hbx9OR9Wm4l0ZPgheip0Ai8zWjAXaM0au/OhGbYrss7ilRySYL
 ORUZqCp0/972w7oyHeHeNu2gxvbg8ouYCSTEftF4v78UPToYmoVYjUHVgxyLEeQ4
 i+B/K2sa96WYa0D64T3cDi7Vmf61ouB86Do7iIU8LZJuiS18Smyz5dfAUNCWZ7br
 rREDnJ6QPjoWMpdz/toYkYxChBWN7td8Zve1u0LUEFXt/OG6jpfv4Dg2jJ+j5E4V
 TGmV5p+7YVpuxmi1ab/MEOTWdG1rckhr85/+Fxy55V+BH8GaU5GEzeFg/5gIwypp
 it4dbPRq/fUPmuuWA+o2BYDC3ytJuFP/UQkCvxazT1OQS5xSHFHZXQYKOkRGZWln
 jNGML2CTTRe4JvRDU1rlLtQjSjlyF9Gt/5fRYkJQsJYByJhpvgp1+Fsuv8e2TFGX
 hhPH/RJJ04nhyHMjH6WBea+3BiUDHk9nxTPxeefu3tpF779vYmBXgO70thYt+J7d
 oJwegyfwJmgv7ZyItgQC+oo9AOVyByXrELuPgbFxHCnxNXjE8Xzu7lxjI3SX1zn2
 1tFX7yqJamR0s7EFwMDVBsG2SqUr/XKmruEUxt+VWIM/UjDrVvDPnqQ0koziRNV6
 dWClV/+QuOXKo349bDQUMjUp7gBub6lHCIJ2U428Y/pckgsOWac=
 =Jun4
 -----END PGP SIGNATURE-----

Merge tag 'dmaengine-5.8-rc1' of git://git.infradead.org/users/vkoul/slave-dma

Pull dmaengine updates from Vinod Koul:
 "A fairly small dmaengine update which includes mostly driver updates
  (dmatest, dw-edma, ioat, mmp-tdma and k3-udma) along with Renesas
  binding update to json-schema"

* tag 'dmaengine-5.8-rc1' of git://git.infradead.org/users/vkoul/slave-dma: (39 commits)
  dmaengine: imx-sdma: initialize all script addresses
  dmaengine: ti: k3-udma: Use proper return code in alloc_chan_resources
  dmaengine: ti: k3-udma: Remove udma_chan.in_ring_cnt
  dmaengine: ti: k3-udma: Add missing dma_sync call for rx flush descriptor
  dmaengine: at_xdmac: Replace zero-length array with flexible-array
  dmaengine: at_hdmac: Replace zero-length array with flexible-array
  dmaengine: qcom: bam_dma: Replace zero-length array with flexible-array
  dmaengine: ti: k3-udma: Use PTR_ERR_OR_ZERO() to simplify code
  dmaengine: moxart-dma: Drop pointless static qualifier in moxart_probe()
  dmaengine: sf-pdma: Simplify the error handling path in 'sf_pdma_probe()'
  dmaengine: qcom_hidma: use true,false for bool variable
  dmaengine: dw-edma: support local dma device transfer semantics
  dmaengine: Fix doc strings to satisfy validation script
  dmaengine: Include dmaengine.h into dmaengine.c
  dmaengine: dmatest: Describe members of struct dmatest_info
  dmaengine: dmatest: Describe members of struct dmatest_params
  dmaengine: dmatest: Allow negative timeout value to specify infinite wait
  Revert "dmaengine: dmatest: timeout value of -1 should specify infinite wait"
  dmaengine: stm32-dma: direct mode support through device tree
  dt-bindings: dma: add direct mode support through device tree in stm32-dma
  ...
2020-06-10 11:03:04 -07:00
Dave Jiang
4f302642b7 dmaengine: idxd: fix interrupt completion after unmasking
The current implementation may miss completions after we unmask the
interrupt. In order to make sure we process all competions, we need to:
1. Do an MMIO read from the device as a barrier to ensure that all PCI
   writes for completions have arrived.
2. Check for any additional completions that we missed.

Fixes: 8f47d1a5e5 ("dmaengine: idxd: connect idxd to dmaengine subsystem")

Reported-by: Sanjay Kumar <sanjay.k.kumar@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/158834641769.35613.1341160109892008587.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-05-04 14:25:37 +05:30
Dave Jiang
c2ce6bbcfc dmaengine: idxd: export hw version through sysfs
Some user apps would like to know the hardware version in order to
determine the variation of the hardware. Export the hardware version number
to userspace via sysfs.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/158696714008.39484.13401950732606906479.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-04-15 21:54:00 +05:30
Linus Torvalds
e964f1e04a dmaengine updates for v5.7-rc1
- Core:
     - Some code cleanup and optimization in core by Andy
     - Debugfs support for displaying dmaengine channels by Peter
 
   - Drivers:
     - New driver for uniphier-xdmac controller
     - Updates to stm32 dma, mdma and dmamux drivers and PM support
     - More updates to idxd drivers
     - Bunch of changes in tegra-apb driver and cleaning up of pm functions
     - Bunch of spelling fixes and Replace zero-length array patches
     - Shutdown hook for fsl-dpaa2-qdma driver
     - Support for interleaved transfers for ti-edma and virtualization
       support for k3-dma driver
     - Support for reset and updates in xilinx_dma driver
     - Improvements and locking updates in at_hdma driver
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+vs47OPLdNbVcHzyfBQHDyUjg0cFAl6FyokACgkQfBQHDyUj
 g0dnFRAAj9lvpflrL+b9eWBZkY1ElV1jAdxsTs4HnYdXQM3ijw8yOosVVSqiuiOy
 2qMfSRTP7qU9gqZ7oa1fnh05DqPmuTc3OF2IZlvGzkU9CiGQ735WGGGG8FfK/dZe
 F4OgQGwA45b47hNIbvM4acwWZYPL+pBuYusKdjdHkouqVM4SORiNM8aRrCJ59xIn
 P9TER//sMpdMEASuRuUIQnXb+OzSNPn1mLiP3zT0XHSM/nBMTAm7AnCDNT/Tjs9f
 hwk2j8rLrwllHGqeZln8cWLhUCPrZFNe5pBWtWyV3MyY/nxlrcUX0ndJUGJIDtsb
 nfXc4QKemOeF1RsC8DsQ/AY8jl6HFvRzWEEkq742IrLPCu/nTnxia4dbXW9MJ0Dp
 BI7IPwoaOoYqBdRkBnSVS2F4x3813egsEReznlu/sUorTIG2g9sWtmuzv6eRt4ow
 HczGgfdJXfCvIKbRg5TIXpbaJogbbB+1YrUlWq9vrZyhVw0ULtfxlWVKDy5VI1cL
 0Kiz/ZIGuoQ9h6E4G3jCpaQTV49tNbYp+vimU9kizmcm+WXrTXR7rgD4AI5tH2DQ
 pxYXNEl4gm1NRtWL1zzJ+B1C0MPXpc1Xafl92W39D6rphEGOdVVzay8meVIaQKDU
 qQaZ1dEK4uuSxwj8NrF7sXHSClafF888FFJBEMArde1HVql/HRU=
 =+UJ7
 -----END PGP SIGNATURE-----

Merge tag 'dmaengine-5.7-rc1' of git://git.infradead.org/users/vkoul/slave-dma

Pull dmaengine updates from Vinod Koul:
 "Core:
   - Some code cleanup and optimization in core by Andy

   - Debugfs support for displaying dmaengine channels by Peter

  Drivers:
   - New driver for uniphier-xdmac controller

   - Updates to stm32 dma, mdma and dmamux drivers and PM support

   - More updates to idxd drivers

   - Bunch of changes in tegra-apb driver and cleaning up of pm
     functions

   - Bunch of spelling fixes and Replace zero-length array patches

   - Shutdown hook for fsl-dpaa2-qdma driver

   - Support for interleaved transfers for ti-edma and virtualization
     support for k3-dma driver

   - Support for reset and updates in xilinx_dma driver

   - Improvements and locking updates in at_hdma driver"

* tag 'dmaengine-5.7-rc1' of git://git.infradead.org/users/vkoul/slave-dma: (89 commits)
  dt-bindings: dma: renesas,usb-dmac: add r8a77961 support
  dmaengine: uniphier-xdmac: Remove redandant error log for platform_get_irq
  dmaengine: tegra-apb: Improve DMA synchronization
  dmaengine: tegra-apb: Don't save/restore IRQ flags in interrupt handler
  dmaengine: tegra-apb: mark PM functions as __maybe_unused
  dmaengine: fix spelling mistake "exceds" -> "exceeds"
  dmaengine: sprd: Set request pending flag when DMA controller is active
  dmaengine: ppc4xx: Use scnprintf() for avoiding potential buffer overflow
  dmaengine: idxd: remove global token limit check
  dmaengine: idxd: reflect shadow copy of traffic class programming
  dmaengine: idxd: Merge definition of dsa_batch_desc into dsa_hw_desc
  dmaengine: Create debug directories for DMA devices
  dmaengine: ti: k3-udma: Implement custom dbg_summary_show for debugfs
  dmaengine: Add basic debugfs support
  dmaengine: fsl-dpaa2-qdma: remove set but not used variable 'dpaa2_qdma'
  dmaengine: ti: edma: fix null dereference because of a typo in pointer name
  dmaengine: fsl-dpaa2-qdma: Adding shutdown hook
  dmaengine: uniphier-xdmac: Add UniPhier external DMA controller driver
  dt-bindings: dmaengine: Add UniPhier external DMA controller bindings
  dmaengine: ti: k3-udma: Implement support for atype (for virtualization)
  ...
2020-04-02 16:04:42 -07:00
Dave Jiang
988aad2f11 dmaengine: idxd: fix off by one on cdev dwq refcount
The refcount check for dedicated workqueue (dwq) is off by one and allows
more than 1 user to open the char device. Fix check so only a single user
can open the device.

Fixes: 42d279f913 ("dmaengine: idxd: add char driver to expose submission portal to userland")
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/158403020187.10208.14117394394540710774.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-03-23 11:32:06 +05:30
Dave Jiang
91124ac612 dmaengine: idxd: remove global token limit check
The global token_limit is not tied to group tokens_reserved and
tokens_allowed parameters. Remove the check in order to allow independent
configuration.

Fixes: c52ca47823 ("dmaengine: idxd: add configuration component of driver")
Reported-by: Yixin Zhang <yixin.zhang@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/158386266911.11066.7545764533072221536.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-03-11 15:08:52 +05:30
Dave Jiang
a1fcaf07ec dmaengine: idxd: reflect shadow copy of traffic class programming
The traffic class are set to -1 at initialization until the user programs
them. If the user choose not to, the driver will program appropriate
defaults. The driver also needs to update the shadowed copies of the values
after doing the programming.

Fixes: c52ca47823 ("dmaengine: idxd: add configuration component of driver")
Reported-by: Yixin Zhang <yixin.zhang@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/158386263076.10898.4586509576813094559.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-03-11 15:08:52 +05:30
Dave Jiang
9065958ee6 dmaengine: idxd: expose general capabilities register in sysfs
There are some capabilities for the device that are interesting to user
apps that are interacting directly with the device. Expose gencap register
in sysfs to allow that information.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/158256729399.55526.10842505054968710547.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-03-02 12:43:49 +05:30
Dave Jiang
b391554c61 dmaengine: idxd: check return result from check_vma() in cdev
The returned result from the check_vma() function in the cdev ->mmap() call
needs to be handled. Add the check and returning error.

Fixes: 42d279f913 ("dmaengine: idxd: add char driver to expose submission portal to userland")
Reported-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/158264926659.9387.14325163515683047959.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-03-02 12:43:49 +05:30
Dave Jiang
50e7e7f6f2 dmaengine: idxd: wq size configuration needs to check global max size
The current size_store() function for idxd sysfs does not check the total
wq size. This allows configuration of all wqs with total wq size. Add check
to make sure the wq sysfs attribute rejects storing of size over the total
wq size.

Fixes: c52ca47823 ("dmaengine: idxd: add configuration component of driver")
Reported-by: Jerry Chen <jerry.t.chen@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/158213309629.2509.3583411832507185041.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-02-24 22:02:34 +05:30
Dave Jiang
88402c5b1b dmaengine: idxd: sysfs input of wq incorrect wq type should return error
Currently when inputing an unrecognized wq type, we set the wq type to
"none". It really should return error and not change the existing wq type
that's in the kernel.

Fixes: c52ca47823 ("dmaengine: idxd: add configuration component of driver")
Reported-by: Yixin Zhang <yixin.zhang@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/158213304803.2290.13336343633425868211.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-02-24 22:02:34 +05:30
Dave Jiang
2d0b191945 dmaengine: idxd: correct reserved token calculation
The calcuation for limit of reserved token did not take into account the
change the user wanted vs the current group reserved token. This causes
changing of the reserved token to be possible only after we set the value
of the reserved token back to 0. Fix calculation so we can set a value that
is non zero for reserved token.

Fixes: c52ca47823 ("dmaengine: idxd: add configuration component of driver")
Reported-by: Jerry Chen <jerry.t.chen@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/158204471889.37789.7749177228265869168.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-02-19 14:48:46 +05:30
Dan Carpenter
2227ab4216 dmaengine: idxd: Fix error handling in idxd_wq_cdev_dev_setup()
We can't call kfree(dev) after calling device_register(dev).  The "dev"
pointer has to be freed using put_device().

Fixes: 42d279f913 ("dmaengine: idxd: add char driver to expose submission portal to userland")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/20200205123248.hmtog7qa2eiqaagh@kili.mountain
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-02-13 20:37:23 +05:30
Dave Jiang
61b5865d56 dmaengine: idxd: fix runaway module ref count on device driver bind
idxd_config_bus_probe() calls try_module_get() but never calls module_put()
when it fails. Thus with every failed attempt, the ref count goes up. Add
module_put() in failure paths.

Fixes: c52ca47823 ("dmaengine: idxd: add configuration component of driver")
Reported-by: Jerry Chen <jerry.t.chen@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/158144296730.41381.12134210685456322434.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-02-13 20:11:00 +05:30
YueHaibing
bfc8f1a87c dmaengine: idxd: remove set but not used variable 'idxd_cdev'
drivers/dma/idxd/cdev.c: In function idxd_cdev_open:
drivers/dma/idxd/cdev.c:77:20: warning:
 variable idxd_cdev set but not used [-Wunused-but-set-variable]

commit 42d279f913 ("dmaengine: idxd: add char driver to
expose submission portal to userland") involed this.

Reported-by: Hulk Robot <hulkci@huawei.com>
Signed-off-by: YueHaibing <yuehaibing@huawei.com>
Acked-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/20200210151855.55044-1-yuehaibing@huawei.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2020-02-13 20:03:41 +05:30