AcoSail/kernel_optimize_test
8
0
Fork 0
forked from luck/tmp_suning_uos_patched
Code Pull Requests Activity
b2afb64ccc
kernel_optimize_test/arch/mips/kernel/pm-cps.c

739 lines
21 KiB
C
Raw Normal View History

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 Based on 1 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license as published by the free software foundation either version 2 of the license or at your option any later version extracted by the scancode license scanner the SPDX license identifier GPL-2.0-or-later has been chosen to replace the boilerplate/reference in 3029 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Allison Randal <allison@lohutok.net> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190527070032.746973796@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-05-27 14:55:01 +08:00
// SPDX-License-Identifier: GPL-2.0-or-later
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
/*
* Copyright (C) 2014 Imagination Technologies
Update MIPS email addresses MIPS will soon not be a part of Imagination Technologies, and as such many @imgtec.com email addresses will no longer be valid. This patch updates the addresses for those who: - Have 10 or more patches in mainline authored using an @imgtec.com email address, or any patches dated within the past year. - Are still with Imagination but leaving as part of the MIPS business unit, as determined from an internal email address list. - Haven't already updated their email address (ie. JamesH) or expressed a desire to be excluded (ie. Maciej). - Acked v2 or earlier of this patch, which leaves Deng-Cheng, Matt & myself. New addresses are of the form firstname.lastname@mips.com, and all verified against an internal email address list. An entry is added to .mailmap for each person such that get_maintainer.pl will report the new addresses rather than @imgtec.com addresses which will soon be dead. Instances of the affected addresses throughout the tree are then mechanically replaced with the new @mips.com address. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: Deng-Cheng Zhu <dengcheng.zhu@imgtec.com> Cc: Deng-Cheng Zhu <dengcheng.zhu@mips.com> Acked-by: Dengcheng Zhu <dengcheng.zhu@mips.com> Cc: Matt Redfearn <matt.redfearn@imgtec.com> Cc: Matt Redfearn <matt.redfearn@mips.com> Acked-by: Matt Redfearn <matt.redfearn@mips.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: linux-kernel@vger.kernel.org Cc: linux-mips@linux-mips.org Cc: trivial@kernel.org Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2017-10-26 08:04:33 +08:00
* Author: Paul Burton <paul.burton@mips.com>
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
*/
MIPS: pm-cps: Generate idle state entry code when CPUs are onlined The MIPS Coherent Processing System (CPS) power management code has previously generated code used to enter low power idle states once during boot for all CPUs. This has the drawback that if a CPU is present in the system but not being used (for example due to the maxcpus kernel parameter) then we encounter problems due to not having probed that CPU for information about its type & properties. The result of this is that we generate entry code which is both unused, potentially entirely invalid & likely to be unsuitable for the CPU in question anyway. Avoid this by generating idle state entry code only when a CPU is brought online. This way we only ever generate code for CPUs that we know we've probed the properties of, and that will actually be used. [ralf@linux-mips.org: Resolve merge conflict.] Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14259/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-14 18:00:27 +08:00
#include <linux/cpuhotplug.h>
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
#include <linux/init.h>
#include <linux/percpu.h>
#include <linux/slab.h>
MIPS: pm-cps: Block system suspend when a JTAG probe is present If a JTAG probe is connected to a MIPS cluster, then the CPC detects it and latches the CPC.STAT_CONF.EJTAG_PROBE bit to 1. While set, attempting to send a power-down command to a core will be blocked, and the CPC will instead send the core to clock-off state. This can interfere with systems fully entering a low power state where all cores, CM, GIC, etc are powered down. Detect that a JTAG probe is / has been connected to the cluster and block the suspend attempt. Attempting to suspend the system while a JTAG probe is connected now yields: # echo mem > /sys/power/state [ 11.654000] PM: Syncing filesystems ... done. [ 11.658000] JTAG probe is connected - abort suspend -sh: echo: write error: Operation not permitted # To restore suspend, the JTAG probe should be disconnected or put into quiescent state. Platform code can then clear the CPC.STAT_CONF.EJTAG_PROBE bit. Reported-by: Ed Blake <ed.blake@sondrel.com> Signed-off-by: Matt Redfearn <matt.redfearn@mips.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Paul Burton <paul.burton@mips.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/18641/ Signed-off-by: James Hogan <jhogan@kernel.org>
2018-02-20 17:58:16 +08:00
#include <linux/suspend.h>
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
#include <asm/asm-offsets.h>
#include <asm/cacheflush.h>
#include <asm/cacheops.h>
#include <asm/idle.h>
MIPS: CPS: Have asm/mips-cps.h include CM & CPC headers With Coherence Manager (CM) 3.5 information about the topology of the system, which has previously only been available through & accessed from the CM, is now also provided by the Cluster Power Controller (CPC). This includes a new CPC_CONFIG register mirroring GCR_CONFIG, and similarly a new CPC_Cx_CONFIG register mirroring GCR_Cx_CONFIG. In preparation for adjusting functions such as mips_cm_numcores(), which have previously only needed to access the CM, to also access the CPC this patch modifies the way we use the various CPS headers. Rather than having users include asm/mips-cm.h or asm/mips-cpc.h individually we instead have users include asm/mips-cps.h which in turn includes asm/mips-cm.h & asm/mips-cpc.h. This means that users will gain access to both CM & CPC registers by including one header, and most importantly it makes asm/mips-cps.h an ideal location for helper functions which need to access the various components of the CPS. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/17015/ Patchwork: https://patchwork.linux-mips.org/patch/17217/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2017-08-13 10:49:41 +08:00
#include <asm/mips-cps.h>
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
#include <asm/mipsmtregs.h>
#include <asm/pm.h>
#include <asm/pm-cps.h>
#include <asm/smp-cps.h>
#include <asm/uasm.h>
/*
* cps_nc_entry_fn - type of a generated non-coherent state entry function
* @online: the count of online coupled VPEs
* @nc_ready_count: pointer to a non-coherent mapping of the core ready_count
*
* The code entering & exiting non-coherent states is generated at runtime
* using uasm, in order to ensure that the compiler cannot insert a stray
* memory access at an unfortunate time and to allow the generation of optimal
* core-specific code particularly for cache routines. If coupled_coherence
* is non-zero and this is the entry function for the CPS_PM_NC_WAIT state,
* returns the number of VPEs that were in the wait state at the point this
* VPE left it. Returns garbage if coupled_coherence is zero or this is not
* the entry function for CPS_PM_NC_WAIT.
*/
typedef unsigned (*cps_nc_entry_fn)(unsigned online, u32 *nc_ready_count);
/*
* The entry point of the generated non-coherent idle state entry/exit
* functions. Actually per-core rather than per-CPU.
*/
static DEFINE_PER_CPU_READ_MOSTLY(cps_nc_entry_fn[CPS_PM_STATE_COUNT],
nc_asm_enter);
/* Bitmap indicating which states are supported by the system */
MIPS: Declare various variables & functions static We currently have various variables & functions which are only used within a single translation unit, but which we don't declare static. This causes various sparse warnings of the form: arch/mips/kernel/mips-r2-to-r6-emul.c:49:1: warning: symbol 'mipsr2emustats' was not declared. Should it be static? arch/mips/kernel/unaligned.c:1381:11: warning: symbol 'reg16to32st' was not declared. Should it be static? arch/mips/mm/mmap.c:146:15: warning: symbol 'arch_mmap_rnd' was not declared. Should it be static? Fix these & others by declaring various affected variables & functions static, avoiding the sparse warnings & redundant symbols. [ralf@linux-mips.org: Add Marcin's build fix.] Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Cc: trivial@kernel.org Patchwork: https://patchwork.linux-mips.org/patch/17176/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2017-08-24 02:17:54 +08:00
static DECLARE_BITMAP(state_support, CPS_PM_STATE_COUNT);
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
/*
* Indicates the number of coupled VPEs ready to operate in a non-coherent
* state. Actually per-core rather than per-CPU.
*/
static DEFINE_PER_CPU_ALIGNED(u32*, ready_count);
/* Indicates online CPUs coupled with the current CPU */
static DEFINE_PER_CPU_ALIGNED(cpumask_t, online_coupled);
/*
* Used to synchronize entry to deep idle states. Actually per-core rather
* than per-CPU.
*/
static DEFINE_PER_CPU_ALIGNED(atomic_t, pm_barrier);
/* Saved CPU state across the CPS_PM_POWER_GATED state */
DEFINE_PER_CPU_ALIGNED(struct mips_static_suspend_state, cps_cpu_state);
/* A somewhat arbitrary number of labels & relocs for uasm */
MIPS: pm-cps: Generate idle state entry code when CPUs are onlined The MIPS Coherent Processing System (CPS) power management code has previously generated code used to enter low power idle states once during boot for all CPUs. This has the drawback that if a CPU is present in the system but not being used (for example due to the maxcpus kernel parameter) then we encounter problems due to not having probed that CPU for information about its type & properties. The result of this is that we generate entry code which is both unused, potentially entirely invalid & likely to be unsuitable for the CPU in question anyway. Avoid this by generating idle state entry code only when a CPU is brought online. This way we only ever generate code for CPUs that we know we've probed the properties of, and that will actually be used. [ralf@linux-mips.org: Resolve merge conflict.] Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14259/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-14 18:00:27 +08:00
static struct uasm_label labels[32];
static struct uasm_reloc relocs[32];
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
enum mips_reg {
zero, at, v0, v1, a0, a1, a2, a3,
t0, t1, t2, t3, t4, t5, t6, t7,
s0, s1, s2, s3, s4, s5, s6, s7,
t8, t9, k0, k1, gp, sp, fp, ra,
};
bool cps_pm_support_state(enum cps_pm_state state)
{
return test_bit(state, state_support);
}
static void coupled_barrier(atomic_t *a, unsigned online)
{
/*
* This function is effectively the same as
* cpuidle_coupled_parallel_barrier, which can't be used here since
* there's no cpuidle device.
*/
if (!coupled_coherence)
return;
MIPS: pm-cps: convert smp_mb__*() Commit 91bbefe6b0fc "arch,mips: Convert smp_mb__*()" replaced the smp_mb__* functions with a simpler API, whilst commit 3179d37ee1ed "MIPS: pm-cps: add PM state entry code for CPS systems" introduced new uses of smp_mb__before_atomic_inc & smp_mb__after_clear_bit. Replace those calls with the corresponding before & after atomic functions of the new, simpler API in order to avoid a build failure: arch/mips/kernel/pm-cps.c: In function 'coupled_barrier': arch/mips/kernel/pm-cps.c:104:2: error: 'smp_mb__before_atomic_inc' is deprecated (declared at include/linux/atomic.h:11) [-Werror=deprecated-declarations] arch/mips/kernel/pm-cps.c: In function 'cps_pm_enter_state': arch/mips/kernel/pm-cps.c:161:2: error: 'smp_mb__after_clear_bit' is deprecated (declared at include/linux/bitops.h:48) [-Werror=deprecated-declarations] Signed-off-by: Paul Burton <paul.burton@imgtec.com> Reviewed-by: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/7086/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2014-06-11 18:00:57 +08:00
smp_mb__before_atomic();
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
atomic_inc(a);
while (atomic_read(a) < online)
cpu_relax();
if (atomic_inc_return(a) == online * 2) {
atomic_set(a, 0);
return;
}
while (atomic_read(a) > online)
cpu_relax();
}
int cps_pm_enter_state(enum cps_pm_state state)
{
unsigned cpu = smp_processor_id();
MIPS: Abstract CPU core & VP(E) ID access through accessor functions We currently have fields in struct cpuinfo_mips for the core & VP(E) ID of a particular CPU, and various pieces of code directly access those fields. This patch abstracts such access by introducing accessor functions cpu_core(), cpu_set_core(), cpu_vpe_id() & cpu_set_vpe_id() and having code that needs to access these values call those functions rather than directly accessing the struct cpuinfo_mips fields. This prepares us for changes to the way in which those values are stored in later patches. The cpu_vpe_id() function is introduced even though we already had a cpu_vpe_id() macro for a couple of reasons: 1) It's more consistent with the core, and future cluster, accessors. 2) It ensures a sensible return type without explicit casts. 3) It's generally preferable to use functions rather than macros. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/17009/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2017-08-13 10:49:35 +08:00
unsigned core = cpu_core(&current_cpu_data);
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
unsigned online, left;
cpumask_t *coupled_mask = this_cpu_ptr(&online_coupled);
u32 *core_ready_count, *nc_core_ready_count;
void *nc_addr;
cps_nc_entry_fn entry;
struct core_boot_config *core_cfg;
struct vpe_boot_config *vpe_cfg;
/* Check that there is an entry function for this state */
entry = per_cpu(nc_asm_enter, core)[state];
if (!entry)
return -EINVAL;
/* Calculate which coupled CPUs (VPEs) are online */
MIPS: pm-cps: Add MIPSr6 CPU support This patch adds support for CPUs implementing the MIPSr6 ISA to the CPS power management code. Three changes are necessary: 1. In MIPSr6, coupled coherence is necessary when CPUS implement multiple Virtual Processors (VPs). 2. MIPSr6 virtual processors are more like real cores and cannot yield to other VPs on the same core, so drop the MT ASE yield instruction. 3. To halt a MIPSr6 VP, the CPC VP_STOP register is used rather than the MT ASE TCHalt CP0 register. Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com> Reviewed-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Kees Cook <keescook@chromium.org> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14225/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-07 17:45:17 +08:00
#if defined(CONFIG_MIPS_MT) || defined(CONFIG_CPU_MIPSR6)
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
if (cpu_online(cpu)) {
cpumask_and(coupled_mask, cpu_online_mask,
&cpu_sibling_map[cpu]);
online = cpumask_weight(coupled_mask);
cpumask_clear_cpu(cpu, coupled_mask);
} else
#endif
{
cpumask_clear(coupled_mask);
online = 1;
}
/* Setup the VPE to run mips_cps_pm_restore when started again */
tree-wide: replace config_enabled() with IS_ENABLED() The use of config_enabled() against config options is ambiguous. In practical terms, config_enabled() is equivalent to IS_BUILTIN(), but the author might have used it for the meaning of IS_ENABLED(). Using IS_ENABLED(), IS_BUILTIN(), IS_MODULE() etc. makes the intention clearer. This commit replaces config_enabled() with IS_ENABLED() where possible. This commit is only touching bool config options. I noticed two cases where config_enabled() is used against a tristate option: - config_enabled(CONFIG_HWMON) [ drivers/net/wireless/ath/ath10k/thermal.c ] - config_enabled(CONFIG_BACKLIGHT_CLASS_DEVICE) [ drivers/gpu/drm/gma500/opregion.c ] I did not touch them because they should be converted to IS_BUILTIN() in order to keep the logic, but I was not sure it was the authors' intention. Link: http://lkml.kernel.org/r/1465215656-20569-1-git-send-email-yamada.masahiro@socionext.com Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Kees Cook <keescook@chromium.org> Cc: Stas Sergeev <stsp@list.ru> Cc: Matt Redfearn <matt.redfearn@imgtec.com> Cc: Joshua Kinard <kumba@gentoo.org> Cc: Jiri Slaby <jslaby@suse.com> Cc: Bjorn Helgaas <bhelgaas@google.com> Cc: Borislav Petkov <bp@suse.de> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: "Dmitry V. Levin" <ldv@altlinux.org> Cc: yu-cheng yu <yu-cheng.yu@intel.com> Cc: James Hogan <james.hogan@imgtec.com> Cc: Brian Gerst <brgerst@gmail.com> Cc: Johannes Berg <johannes@sipsolutions.net> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Al Viro <viro@zeniv.linux.org.uk> Cc: Will Drewry <wad@chromium.org> Cc: Nikolay Martynov <mar.kolya@gmail.com> Cc: Huacai Chen <chenhc@lemote.com> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Daniel Borkmann <daniel@iogearbox.net> Cc: Leonid Yegoshin <Leonid.Yegoshin@imgtec.com> Cc: Rafal Milecki <zajec5@gmail.com> Cc: James Cowgill <James.Cowgill@imgtec.com> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Alex Smith <alex.smith@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Qais Yousef <qais.yousef@imgtec.com> Cc: Jiang Liu <jiang.liu@linux.intel.com> Cc: Mikko Rapeli <mikko.rapeli@iki.fi> Cc: Paul Gortmaker <paul.gortmaker@windriver.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: Brian Norris <computersforpeace@gmail.com> Cc: Hidehiro Kawai <hidehiro.kawai.ez@hitachi.com> Cc: "Luis R. Rodriguez" <mcgrof@do-not-panic.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Ingo Molnar <mingo@redhat.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> Cc: Roland McGrath <roland@hack.frob.com> Cc: Paul Burton <paul.burton@imgtec.com> Cc: Kalle Valo <kvalo@qca.qualcomm.com> Cc: Viresh Kumar <viresh.kumar@linaro.org> Cc: Tony Wu <tung7970@gmail.com> Cc: Huaitong Han <huaitong.han@intel.com> Cc: Sumit Semwal <sumit.semwal@linaro.org> Cc: Alexei Starovoitov <ast@kernel.org> Cc: Juergen Gross <jgross@suse.com> Cc: Jason Cooper <jason@lakedaemon.net> Cc: "David S. Miller" <davem@davemloft.net> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Andrea Gelmini <andrea.gelmini@gelma.net> Cc: David Woodhouse <dwmw2@infradead.org> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Rabin Vincent <rabin@rab.in> Cc: "Maciej W. Rozycki" <macro@imgtec.com> Cc: David Daney <david.daney@cavium.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-08-04 04:45:50 +08:00
if (IS_ENABLED(CONFIG_CPU_PM) && state == CPS_PM_POWER_GATED) {
MIPS: pm-cps: Prevent use of mips_cps_* without CPS SMP These symbols will not be defined when CONFIG_MIPS_CPS=n, but although the CPS_PM_POWER_GATED state will never be used in that case the compiler doesn't have enough information to figure that out. Add checks which evaluate to a constant false for CONFIG_MIPS_CPS=n cases in order to help the compiler out & eliminate the symbol references. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Reviewed-by: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/7278/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2014-07-09 19:48:18 +08:00
/* Power gating relies upon CPS SMP */
if (!mips_cps_smp_in_use())
return -EINVAL;
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
core_cfg = &mips_cps_core_bootcfg[core];
MIPS: {pm,smp}-cps: use cpu_vpe_id macro When determining the VPE ID of a CPU, make use of the cpu_vpe_id macro which will return 0 in a non-MT kernel build. Most code is already doing so but a couple of places weren't. Fixing this prevents a build failure for non-MT kernels where struct cpuinfo_mips does not contain the vpe_id field: arch/mips/kernel/pm-cps.c: In function 'cps_pm_enter_state': arch/mips/kernel/pm-cps.c:153:51: error: 'struct cpuinfo_mips' has no member named 'vpe_id' vpe_cfg = &core_cfg->vpe_config[current_cpu_data.vpe_id]; arch/mips/kernel/smp-cps.c: In function 'wait_for_sibling_halt': arch/mips/kernel/smp-cps.c:363:33: error: 'struct cpuinfo_mips' has no member named 'vpe_id' unsigned vpe_id = cpu_data[cpu].vpe_id; Signed-off-by: Paul Burton <paul.burton@imgtec.com> Reviewed-by: Markos Chandras <markos.chandras@imgtec.com> Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2014-07-09 19:48:21 +08:00
vpe_cfg = &core_cfg->vpe_config[cpu_vpe_id(&current_cpu_data)];
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
vpe_cfg->pc = (unsigned long)mips_cps_pm_restore;
vpe_cfg->gp = (unsigned long)current_thread_info();
vpe_cfg->sp = 0;
}
/* Indicate that this CPU might not be coherent */
cpumask_clear_cpu(cpu, &cpu_coherent_mask);
MIPS: pm-cps: convert smp_mb__*() Commit 91bbefe6b0fc "arch,mips: Convert smp_mb__*()" replaced the smp_mb__* functions with a simpler API, whilst commit 3179d37ee1ed "MIPS: pm-cps: add PM state entry code for CPS systems" introduced new uses of smp_mb__before_atomic_inc & smp_mb__after_clear_bit. Replace those calls with the corresponding before & after atomic functions of the new, simpler API in order to avoid a build failure: arch/mips/kernel/pm-cps.c: In function 'coupled_barrier': arch/mips/kernel/pm-cps.c:104:2: error: 'smp_mb__before_atomic_inc' is deprecated (declared at include/linux/atomic.h:11) [-Werror=deprecated-declarations] arch/mips/kernel/pm-cps.c: In function 'cps_pm_enter_state': arch/mips/kernel/pm-cps.c:161:2: error: 'smp_mb__after_clear_bit' is deprecated (declared at include/linux/bitops.h:48) [-Werror=deprecated-declarations] Signed-off-by: Paul Burton <paul.burton@imgtec.com> Reviewed-by: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/7086/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2014-06-11 18:00:57 +08:00
smp_mb__after_atomic();
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
/* Create a non-coherent mapping of the core ready_count */
core_ready_count = per_cpu(ready_count, core);
nc_addr = kmap_noncoherent(virt_to_page(core_ready_count),
(unsigned long)core_ready_count);
nc_addr += ((unsigned long)core_ready_count & ~PAGE_MASK);
nc_core_ready_count = nc_addr;
/* Ensure ready_count is zero-initialised before the assembly runs */
locking/atomics: COCCINELLE/treewide: Convert trivial ACCESS_ONCE() patterns to READ_ONCE()/WRITE_ONCE() Please do not apply this to mainline directly, instead please re-run the coccinelle script shown below and apply its output. For several reasons, it is desirable to use {READ,WRITE}_ONCE() in preference to ACCESS_ONCE(), and new code is expected to use one of the former. So far, there's been no reason to change most existing uses of ACCESS_ONCE(), as these aren't harmful, and changing them results in churn. However, for some features, the read/write distinction is critical to correct operation. To distinguish these cases, separate read/write accessors must be used. This patch migrates (most) remaining ACCESS_ONCE() instances to {READ,WRITE}_ONCE(), using the following coccinelle script: ---- // Convert trivial ACCESS_ONCE() uses to equivalent READ_ONCE() and // WRITE_ONCE() // $ make coccicheck COCCI=/home/mark/once.cocci SPFLAGS="--include-headers" MODE=patch virtual patch @ depends on patch @ expression E1, E2; @@ - ACCESS_ONCE(E1) = E2 + WRITE_ONCE(E1, E2) @ depends on patch @ expression E; @@ - ACCESS_ONCE(E) + READ_ONCE(E) ---- Signed-off-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: davem@davemloft.net Cc: linux-arch@vger.kernel.org Cc: mpe@ellerman.id.au Cc: shuah@kernel.org Cc: snitzer@redhat.com Cc: thor.thayer@linux.intel.com Cc: tj@kernel.org Cc: viro@zeniv.linux.org.uk Cc: will.deacon@arm.com Link: http://lkml.kernel.org/r/1508792849-3115-19-git-send-email-paulmck@linux.vnet.ibm.com Signed-off-by: Ingo Molnar <mingo@kernel.org>
2017-10-24 05:07:29 +08:00
WRITE_ONCE(*nc_core_ready_count, 0);
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
coupled_barrier(&per_cpu(pm_barrier, core), online);
/* Run the generated entry code */
left = entry(online, nc_core_ready_count);
/* Remove the non-coherent mapping of ready_count */
kunmap_noncoherent();
/* Indicate that this CPU is definitely coherent */
cpumask_set_cpu(cpu, &cpu_coherent_mask);
/*
* If this VPE is the first to leave the non-coherent wait state then
* it needs to wake up any coupled VPEs still running their wait
* instruction so that they return to cpuidle, which can then complete
* coordination between the coupled VPEs & provide the governor with
* a chance to reflect on the length of time the VPEs were in the
* idle state.
*/
if (coupled_coherence && (state == CPS_PM_NC_WAIT) && (left == online))
arch_send_call_function_ipi_mask(coupled_mask);
return 0;
}
MIPS: pm-cps: Generate idle state entry code when CPUs are onlined The MIPS Coherent Processing System (CPS) power management code has previously generated code used to enter low power idle states once during boot for all CPUs. This has the drawback that if a CPU is present in the system but not being used (for example due to the maxcpus kernel parameter) then we encounter problems due to not having probed that CPU for information about its type & properties. The result of this is that we generate entry code which is both unused, potentially entirely invalid & likely to be unsuitable for the CPU in question anyway. Avoid this by generating idle state entry code only when a CPU is brought online. This way we only ever generate code for CPUs that we know we've probed the properties of, and that will actually be used. [ralf@linux-mips.org: Resolve merge conflict.] Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14259/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-14 18:00:27 +08:00
static void cps_gen_cache_routine(u32 **pp, struct uasm_label **pl,
struct uasm_reloc **pr,
const struct cache_desc *cache,
unsigned op, int lbl)
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
{
unsigned cache_size = cache->ways << cache->waybit;
unsigned i;
const unsigned unroll_lines = 32;
/* If the cache isn't present this function has it easy */
if (cache->flags & MIPS_CACHE_NOT_PRESENT)
return;
/* Load base address */
UASM_i_LA(pp, t0, (long)CKSEG0);
/* Calculate end address */
if (cache_size < 0x8000)
uasm_i_addiu(pp, t1, t0, cache_size);
else
UASM_i_LA(pp, t1, (long)(CKSEG0 + cache_size));
/* Start of cache op loop */
uasm_build_label(pl, *pp, lbl);
/* Generate the cache ops */
MIPS: pm-cps: Avoid offset overflow on MIPSr6 This is similar to commit 934c79231c1b ("MIPS: asm: r4kcache: Add MIPS R6 cache unroll functions"). The CACHE instruction has been redefined for MIPSr6 and it reduced its offset field to 8 bits. This leads to micro-assembler field overflow warnings when booting SMP MIPSr6 cores like the following one: Call Trace: [<ffffffff8010af88>] show_stack+0x68/0x88 [<ffffffff8056ddf0>] dump_stack+0x68/0x88 [<ffffffff801305bc>] warn_slowpath_common+0x8c/0xc8 [<ffffffff80130630>] warn_slowpath_fmt+0x38/0x48 [<ffffffff80125814>] build_insn+0x514/0x5c0 [<ffffffff806ee134>] cps_gen_cache_routine.isra.3+0xe0/0x1b8 [<ffffffff806ee570>] cps_pm_init+0x364/0x9ec [<ffffffff80100538>] do_one_initcall+0x90/0x1a8 [<ffffffff806e8c14>] kernel_init_freeable+0x160/0x21c [<ffffffff8056b6a0>] kernel_init+0x10/0xf8 [<ffffffff801059f8>] ret_from_kernel_thread+0x14/0x1c We fix this by incrementing the base register on every loop. Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/12329/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-02-03 11:15:23 +08:00
for (i = 0; i < unroll_lines; i++) {
if (cpu_has_mips_r6) {
uasm_i_cache(pp, op, 0, t0);
uasm_i_addiu(pp, t0, t0, cache->linesz);
} else {
uasm_i_cache(pp, op, i * cache->linesz, t0);
}
}
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
MIPS: pm-cps: Avoid offset overflow on MIPSr6 This is similar to commit 934c79231c1b ("MIPS: asm: r4kcache: Add MIPS R6 cache unroll functions"). The CACHE instruction has been redefined for MIPSr6 and it reduced its offset field to 8 bits. This leads to micro-assembler field overflow warnings when booting SMP MIPSr6 cores like the following one: Call Trace: [<ffffffff8010af88>] show_stack+0x68/0x88 [<ffffffff8056ddf0>] dump_stack+0x68/0x88 [<ffffffff801305bc>] warn_slowpath_common+0x8c/0xc8 [<ffffffff80130630>] warn_slowpath_fmt+0x38/0x48 [<ffffffff80125814>] build_insn+0x514/0x5c0 [<ffffffff806ee134>] cps_gen_cache_routine.isra.3+0xe0/0x1b8 [<ffffffff806ee570>] cps_pm_init+0x364/0x9ec [<ffffffff80100538>] do_one_initcall+0x90/0x1a8 [<ffffffff806e8c14>] kernel_init_freeable+0x160/0x21c [<ffffffff8056b6a0>] kernel_init+0x10/0xf8 [<ffffffff801059f8>] ret_from_kernel_thread+0x14/0x1c We fix this by incrementing the base register on every loop. Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/12329/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-02-03 11:15:23 +08:00
if (!cpu_has_mips_r6)
/* Update the base address */
uasm_i_addiu(pp, t0, t0, unroll_lines * cache->linesz);
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
/* Loop if we haven't reached the end address yet */
uasm_il_bne(pp, pr, t0, t1, lbl);
uasm_i_nop(pp);
}
MIPS: pm-cps: Generate idle state entry code when CPUs are onlined The MIPS Coherent Processing System (CPS) power management code has previously generated code used to enter low power idle states once during boot for all CPUs. This has the drawback that if a CPU is present in the system but not being used (for example due to the maxcpus kernel parameter) then we encounter problems due to not having probed that CPU for information about its type & properties. The result of this is that we generate entry code which is both unused, potentially entirely invalid & likely to be unsuitable for the CPU in question anyway. Avoid this by generating idle state entry code only when a CPU is brought online. This way we only ever generate code for CPUs that we know we've probed the properties of, and that will actually be used. [ralf@linux-mips.org: Resolve merge conflict.] Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14259/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-14 18:00:27 +08:00
static int cps_gen_flush_fsb(u32 **pp, struct uasm_label **pl,
struct uasm_reloc **pr,
const struct cpuinfo_mips *cpu_info,
int lbl)
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
{
unsigned i, fsb_size = 8;
unsigned num_loads = (fsb_size * 3) / 2;
unsigned line_stride = 2;
unsigned line_size = cpu_info->dcache.linesz;
unsigned perf_counter, perf_event;
unsigned revision = cpu_info->processor_id & PRID_REV_MASK;
/*
* Determine whether this CPU requires an FSB flush, and if so which
* performance counter/event reflect stalls due to a full FSB.
*/
switch (__get_cpu_type(cpu_info->cputype)) {
case CPU_INTERAPTIV:
perf_counter = 1;
perf_event = 51;
break;
case CPU_PROAPTIV:
/* Newer proAptiv cores don't require this workaround */
if (revision >= PRID_REV_ENCODE_332(1, 1, 0))
return 0;
/* On older ones it's unavailable */
return -1;
default:
MIPS: pm-cps: Change FSB workaround to CPU blacklist The check for whether a CPU required the FSB flush workaround previously required every CPU not requiring it to be whitelisted. That approach does not scale well as new CPUs are introduced so change the default from a WARN and returning an error to just returning 0. Any CPUs requiring the workaround can then be added to the blacklist. Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com> Reviewed-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14218/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-07 17:45:11 +08:00
/* Assume that the CPU does not need this workaround */
return 0;
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
}
/*
* Ensure that the fill/store buffer (FSB) is not holding the results
* of a prefetch, since if it is then the CPC sequencer may become
* stuck in the D3 (ClrBus) state whilst entering a low power state.
*/
/* Preserve perf counter setup */
uasm_i_mfc0(pp, t2, 25, (perf_counter * 2) + 0); /* PerfCtlN */
uasm_i_mfc0(pp, t3, 25, (perf_counter * 2) + 1); /* PerfCntN */
/* Setup perf counter to count FSB full pipeline stalls */
uasm_i_addiu(pp, t0, zero, (perf_event << 5) | 0xf);
uasm_i_mtc0(pp, t0, 25, (perf_counter * 2) + 0); /* PerfCtlN */
uasm_i_ehb(pp);
uasm_i_mtc0(pp, zero, 25, (perf_counter * 2) + 1); /* PerfCntN */
uasm_i_ehb(pp);
/* Base address for loads */
UASM_i_LA(pp, t0, (long)CKSEG0);
/* Start of clear loop */
uasm_build_label(pl, *pp, lbl);
/* Perform some loads to fill the FSB */
for (i = 0; i < num_loads; i++)
uasm_i_lw(pp, zero, i * line_size * line_stride, t0);
/*
* Invalidate the new D-cache entries so that the cache will need
* refilling (via the FSB) if the loop is executed again.
*/
for (i = 0; i < num_loads; i++) {
uasm_i_cache(pp, Hit_Invalidate_D,
i * line_size * line_stride, t0);
uasm_i_cache(pp, Hit_Writeback_Inv_SD,
i * line_size * line_stride, t0);
}
MIPS: pm-cps: Update comments on barrier instructions This code makes large use of barriers, which had quite vague descriptions. Update the comments to make the choice of barrier and reason for it more clear. Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com> Reviewed-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Kees Cook <keescook@chromium.org> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14220/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-07 17:45:12 +08:00
/* Barrier ensuring previous cache invalidates are complete */
MIPS: barrier: Add __SYNC() infrastructure Introduce an asm/sync.h header which provides infrastructure that can be used to generate sync instructions of various types, and for various reasons. For example if we need a sync instruction that provides a full completion barrier but only on systems which have weak memory ordering, we can generate the appropriate assembly code using: __SYNC(full, weak_ordering) When the kernel is configured to run on systems with weak memory ordering (ie. CONFIG_WEAK_ORDERING is selected) we'll emit a sync instruction. When the kernel is configured to run on systems with strong memory ordering (ie. CONFIG_WEAK_ORDERING is not selected) we'll emit nothing. The caller doesn't need to know which happened - it simply says what it needs & when, with no concern for checking the kernel configuration. There are some scenarios in which we may want to emit code only when we *didn't* emit a sync instruction. For example, some Loongson3 CPUs suffer from a bug that requires us to emit a sync instruction prior to each ll instruction (enabled by CONFIG_CPU_LOONGSON3_WORKAROUNDS). In cases where this bug workaround is enabled, it's wasteful to then have more generic code emit another sync instruction to provide barriers we need in general. A __SYNC_ELSE() macro allows for this, providing an extra argument that contains code to be assembled only in cases where the sync instruction was not emitted. For example if we have a scenario in which we generally want to emit a release barrier but for affected Loongson3 configurations upgrade that to a full completion barrier, we can do that like so: __SYNC_ELSE(full, loongson3_war, __SYNC(rl, always)) The assembly generated by these macros can be used either as inline assembly or in assembly source files. Differing types of sync as provided by MIPSr6 are defined, but currently they all generate a full completion barrier except in kernels configured for Cavium Octeon systems. There the wmb sync-type is used, and rmb syncs are omitted, as has been the case since commit 6b07d38aaa52 ("MIPS: Octeon: Use optimized memory barrier primitives."). Using __SYNC() with the wmb or rmb types will abstract away the Octeon specific behavior and allow us to later clean up asm/barrier.h code that currently includes a plethora of #ifdef's. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: linux-mips@vger.kernel.org Cc: Huacai Chen <chenhc@lemote.com> Cc: Jiaxun Yang <jiaxun.yang@flygoat.com> Cc: linux-kernel@vger.kernel.org
2019-10-02 05:53:07 +08:00
uasm_i_sync(pp, __SYNC_full);
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
uasm_i_ehb(pp);
/* Check whether the pipeline stalled due to the FSB being full */
uasm_i_mfc0(pp, t1, 25, (perf_counter * 2) + 1); /* PerfCntN */
/* Loop if it didn't */
uasm_il_beqz(pp, pr, t1, lbl);
uasm_i_nop(pp);
/* Restore perf counter 1. The count may well now be wrong... */
uasm_i_mtc0(pp, t2, 25, (perf_counter * 2) + 0); /* PerfCtlN */
uasm_i_ehb(pp);
uasm_i_mtc0(pp, t3, 25, (perf_counter * 2) + 1); /* PerfCntN */
uasm_i_ehb(pp);
return 0;
}
MIPS: pm-cps: Generate idle state entry code when CPUs are onlined The MIPS Coherent Processing System (CPS) power management code has previously generated code used to enter low power idle states once during boot for all CPUs. This has the drawback that if a CPU is present in the system but not being used (for example due to the maxcpus kernel parameter) then we encounter problems due to not having probed that CPU for information about its type & properties. The result of this is that we generate entry code which is both unused, potentially entirely invalid & likely to be unsuitable for the CPU in question anyway. Avoid this by generating idle state entry code only when a CPU is brought online. This way we only ever generate code for CPUs that we know we've probed the properties of, and that will actually be used. [ralf@linux-mips.org: Resolve merge conflict.] Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14259/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-14 18:00:27 +08:00
static void cps_gen_set_top_bit(u32 **pp, struct uasm_label **pl,
struct uasm_reloc **pr,
unsigned r_addr, int lbl)
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
{
uasm_i_lui(pp, t0, uasm_rel_hi(0x80000000));
uasm_build_label(pl, *pp, lbl);
uasm_i_ll(pp, t1, 0, r_addr);
uasm_i_or(pp, t1, t1, t0);
uasm_i_sc(pp, t1, 0, r_addr);
uasm_il_beqz(pp, pr, t1, lbl);
uasm_i_nop(pp);
}
MIPS: pm-cps: Generate idle state entry code when CPUs are onlined The MIPS Coherent Processing System (CPS) power management code has previously generated code used to enter low power idle states once during boot for all CPUs. This has the drawback that if a CPU is present in the system but not being used (for example due to the maxcpus kernel parameter) then we encounter problems due to not having probed that CPU for information about its type & properties. The result of this is that we generate entry code which is both unused, potentially entirely invalid & likely to be unsuitable for the CPU in question anyway. Avoid this by generating idle state entry code only when a CPU is brought online. This way we only ever generate code for CPUs that we know we've probed the properties of, and that will actually be used. [ralf@linux-mips.org: Resolve merge conflict.] Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14259/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-14 18:00:27 +08:00
static void *cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
{
struct uasm_label *l = labels;
struct uasm_reloc *r = relocs;
u32 *buf, *p;
const unsigned r_online = a0;
const unsigned r_nc_count = a1;
const unsigned r_pcohctl = t7;
const unsigned max_instrs = 256;
unsigned cpc_cmd;
int err;
enum {
lbl_incready = 1,
lbl_poll_cont,
lbl_secondary_hang,
lbl_disable_coherence,
lbl_flush_fsb,
lbl_invicache,
lbl_flushdcache,
lbl_hang,
lbl_set_cont,
lbl_secondary_cont,
lbl_decready,
};
/* Allocate a buffer to hold the generated code */
p = buf = kcalloc(max_instrs, sizeof(u32), GFP_KERNEL);
if (!buf)
return NULL;
/* Clear labels & relocs ready for (re)use */
memset(labels, 0, sizeof(labels));
memset(relocs, 0, sizeof(relocs));
tree-wide: replace config_enabled() with IS_ENABLED() The use of config_enabled() against config options is ambiguous. In practical terms, config_enabled() is equivalent to IS_BUILTIN(), but the author might have used it for the meaning of IS_ENABLED(). Using IS_ENABLED(), IS_BUILTIN(), IS_MODULE() etc. makes the intention clearer. This commit replaces config_enabled() with IS_ENABLED() where possible. This commit is only touching bool config options. I noticed two cases where config_enabled() is used against a tristate option: - config_enabled(CONFIG_HWMON) [ drivers/net/wireless/ath/ath10k/thermal.c ] - config_enabled(CONFIG_BACKLIGHT_CLASS_DEVICE) [ drivers/gpu/drm/gma500/opregion.c ] I did not touch them because they should be converted to IS_BUILTIN() in order to keep the logic, but I was not sure it was the authors' intention. Link: http://lkml.kernel.org/r/1465215656-20569-1-git-send-email-yamada.masahiro@socionext.com Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Kees Cook <keescook@chromium.org> Cc: Stas Sergeev <stsp@list.ru> Cc: Matt Redfearn <matt.redfearn@imgtec.com> Cc: Joshua Kinard <kumba@gentoo.org> Cc: Jiri Slaby <jslaby@suse.com> Cc: Bjorn Helgaas <bhelgaas@google.com> Cc: Borislav Petkov <bp@suse.de> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: "Dmitry V. Levin" <ldv@altlinux.org> Cc: yu-cheng yu <yu-cheng.yu@intel.com> Cc: James Hogan <james.hogan@imgtec.com> Cc: Brian Gerst <brgerst@gmail.com> Cc: Johannes Berg <johannes@sipsolutions.net> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Al Viro <viro@zeniv.linux.org.uk> Cc: Will Drewry <wad@chromium.org> Cc: Nikolay Martynov <mar.kolya@gmail.com> Cc: Huacai Chen <chenhc@lemote.com> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Daniel Borkmann <daniel@iogearbox.net> Cc: Leonid Yegoshin <Leonid.Yegoshin@imgtec.com> Cc: Rafal Milecki <zajec5@gmail.com> Cc: James Cowgill <James.Cowgill@imgtec.com> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Alex Smith <alex.smith@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Qais Yousef <qais.yousef@imgtec.com> Cc: Jiang Liu <jiang.liu@linux.intel.com> Cc: Mikko Rapeli <mikko.rapeli@iki.fi> Cc: Paul Gortmaker <paul.gortmaker@windriver.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: Brian Norris <computersforpeace@gmail.com> Cc: Hidehiro Kawai <hidehiro.kawai.ez@hitachi.com> Cc: "Luis R. Rodriguez" <mcgrof@do-not-panic.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Ingo Molnar <mingo@redhat.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> Cc: Roland McGrath <roland@hack.frob.com> Cc: Paul Burton <paul.burton@imgtec.com> Cc: Kalle Valo <kvalo@qca.qualcomm.com> Cc: Viresh Kumar <viresh.kumar@linaro.org> Cc: Tony Wu <tung7970@gmail.com> Cc: Huaitong Han <huaitong.han@intel.com> Cc: Sumit Semwal <sumit.semwal@linaro.org> Cc: Alexei Starovoitov <ast@kernel.org> Cc: Juergen Gross <jgross@suse.com> Cc: Jason Cooper <jason@lakedaemon.net> Cc: "David S. Miller" <davem@davemloft.net> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Andrea Gelmini <andrea.gelmini@gelma.net> Cc: David Woodhouse <dwmw2@infradead.org> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Rabin Vincent <rabin@rab.in> Cc: "Maciej W. Rozycki" <macro@imgtec.com> Cc: David Daney <david.daney@cavium.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-08-04 04:45:50 +08:00
if (IS_ENABLED(CONFIG_CPU_PM) && state == CPS_PM_POWER_GATED) {
MIPS: pm-cps: Prevent use of mips_cps_* without CPS SMP These symbols will not be defined when CONFIG_MIPS_CPS=n, but although the CPS_PM_POWER_GATED state will never be used in that case the compiler doesn't have enough information to figure that out. Add checks which evaluate to a constant false for CONFIG_MIPS_CPS=n cases in order to help the compiler out & eliminate the symbol references. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Reviewed-by: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/7278/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2014-07-09 19:48:18 +08:00
/* Power gating relies upon CPS SMP */
if (!mips_cps_smp_in_use())
goto out_err;
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
/*
* Save CPU state. Note the non-standard calling convention
* with the return address placed in v0 to avoid clobbering
* the ra register before it is saved.
*/
UASM_i_LA(&p, t0, (long)mips_cps_pm_save);
uasm_i_jalr(&p, v0, t0);
uasm_i_nop(&p);
}
/*
* Load addresses of required CM & CPC registers. This is done early
* because they're needed in both the enable & disable coherence steps
* but in the coupled case the enable step will only run on one VPE.
*/
UASM_i_LA(&p, r_pcohctl, (long)addr_gcr_cl_coherence());
if (coupled_coherence) {
/* Increment ready_count */
MIPS: barrier: Add __SYNC() infrastructure Introduce an asm/sync.h header which provides infrastructure that can be used to generate sync instructions of various types, and for various reasons. For example if we need a sync instruction that provides a full completion barrier but only on systems which have weak memory ordering, we can generate the appropriate assembly code using: __SYNC(full, weak_ordering) When the kernel is configured to run on systems with weak memory ordering (ie. CONFIG_WEAK_ORDERING is selected) we'll emit a sync instruction. When the kernel is configured to run on systems with strong memory ordering (ie. CONFIG_WEAK_ORDERING is not selected) we'll emit nothing. The caller doesn't need to know which happened - it simply says what it needs & when, with no concern for checking the kernel configuration. There are some scenarios in which we may want to emit code only when we *didn't* emit a sync instruction. For example, some Loongson3 CPUs suffer from a bug that requires us to emit a sync instruction prior to each ll instruction (enabled by CONFIG_CPU_LOONGSON3_WORKAROUNDS). In cases where this bug workaround is enabled, it's wasteful to then have more generic code emit another sync instruction to provide barriers we need in general. A __SYNC_ELSE() macro allows for this, providing an extra argument that contains code to be assembled only in cases where the sync instruction was not emitted. For example if we have a scenario in which we generally want to emit a release barrier but for affected Loongson3 configurations upgrade that to a full completion barrier, we can do that like so: __SYNC_ELSE(full, loongson3_war, __SYNC(rl, always)) The assembly generated by these macros can be used either as inline assembly or in assembly source files. Differing types of sync as provided by MIPSr6 are defined, but currently they all generate a full completion barrier except in kernels configured for Cavium Octeon systems. There the wmb sync-type is used, and rmb syncs are omitted, as has been the case since commit 6b07d38aaa52 ("MIPS: Octeon: Use optimized memory barrier primitives."). Using __SYNC() with the wmb or rmb types will abstract away the Octeon specific behavior and allow us to later clean up asm/barrier.h code that currently includes a plethora of #ifdef's. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: linux-mips@vger.kernel.org Cc: Huacai Chen <chenhc@lemote.com> Cc: Jiaxun Yang <jiaxun.yang@flygoat.com> Cc: linux-kernel@vger.kernel.org
2019-10-02 05:53:07 +08:00
uasm_i_sync(&p, __SYNC_mb);
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
uasm_build_label(&l, p, lbl_incready);
uasm_i_ll(&p, t1, 0, r_nc_count);
uasm_i_addiu(&p, t2, t1, 1);
uasm_i_sc(&p, t2, 0, r_nc_count);
uasm_il_beqz(&p, &r, t2, lbl_incready);
uasm_i_addiu(&p, t1, t1, 1);
MIPS: pm-cps: Update comments on barrier instructions This code makes large use of barriers, which had quite vague descriptions. Update the comments to make the choice of barrier and reason for it more clear. Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com> Reviewed-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Kees Cook <keescook@chromium.org> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14220/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-07 17:45:12 +08:00
/* Barrier ensuring all CPUs see the updated r_nc_count value */
MIPS: barrier: Add __SYNC() infrastructure Introduce an asm/sync.h header which provides infrastructure that can be used to generate sync instructions of various types, and for various reasons. For example if we need a sync instruction that provides a full completion barrier but only on systems which have weak memory ordering, we can generate the appropriate assembly code using: __SYNC(full, weak_ordering) When the kernel is configured to run on systems with weak memory ordering (ie. CONFIG_WEAK_ORDERING is selected) we'll emit a sync instruction. When the kernel is configured to run on systems with strong memory ordering (ie. CONFIG_WEAK_ORDERING is not selected) we'll emit nothing. The caller doesn't need to know which happened - it simply says what it needs & when, with no concern for checking the kernel configuration. There are some scenarios in which we may want to emit code only when we *didn't* emit a sync instruction. For example, some Loongson3 CPUs suffer from a bug that requires us to emit a sync instruction prior to each ll instruction (enabled by CONFIG_CPU_LOONGSON3_WORKAROUNDS). In cases where this bug workaround is enabled, it's wasteful to then have more generic code emit another sync instruction to provide barriers we need in general. A __SYNC_ELSE() macro allows for this, providing an extra argument that contains code to be assembled only in cases where the sync instruction was not emitted. For example if we have a scenario in which we generally want to emit a release barrier but for affected Loongson3 configurations upgrade that to a full completion barrier, we can do that like so: __SYNC_ELSE(full, loongson3_war, __SYNC(rl, always)) The assembly generated by these macros can be used either as inline assembly or in assembly source files. Differing types of sync as provided by MIPSr6 are defined, but currently they all generate a full completion barrier except in kernels configured for Cavium Octeon systems. There the wmb sync-type is used, and rmb syncs are omitted, as has been the case since commit 6b07d38aaa52 ("MIPS: Octeon: Use optimized memory barrier primitives."). Using __SYNC() with the wmb or rmb types will abstract away the Octeon specific behavior and allow us to later clean up asm/barrier.h code that currently includes a plethora of #ifdef's. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: linux-mips@vger.kernel.org Cc: Huacai Chen <chenhc@lemote.com> Cc: Jiaxun Yang <jiaxun.yang@flygoat.com> Cc: linux-kernel@vger.kernel.org
2019-10-02 05:53:07 +08:00
uasm_i_sync(&p, __SYNC_mb);
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
/*
* If this is the last VPE to become ready for non-coherence
* then it should branch below.
*/
uasm_il_beq(&p, &r, t1, r_online, lbl_disable_coherence);
uasm_i_nop(&p);
if (state < CPS_PM_POWER_GATED) {
/*
* Otherwise this is not the last VPE to become ready
* for non-coherence. It needs to wait until coherence
* has been disabled before proceeding, which it will do
* by polling for the top bit of ready_count being set.
*/
uasm_i_addiu(&p, t1, zero, -1);
uasm_build_label(&l, p, lbl_poll_cont);
uasm_i_lw(&p, t0, 0, r_nc_count);
uasm_il_bltz(&p, &r, t0, lbl_secondary_cont);
uasm_i_ehb(&p);
MIPS: pm-cps: Add MIPSr6 CPU support This patch adds support for CPUs implementing the MIPSr6 ISA to the CPS power management code. Three changes are necessary: 1. In MIPSr6, coupled coherence is necessary when CPUS implement multiple Virtual Processors (VPs). 2. MIPSr6 virtual processors are more like real cores and cannot yield to other VPs on the same core, so drop the MT ASE yield instruction. 3. To halt a MIPSr6 VP, the CPC VP_STOP register is used rather than the MT ASE TCHalt CP0 register. Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com> Reviewed-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Kees Cook <keescook@chromium.org> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14225/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-07 17:45:17 +08:00
if (cpu_has_mipsmt)
uasm_i_yield(&p, zero, t1);
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
uasm_il_b(&p, &r, lbl_poll_cont);
uasm_i_nop(&p);
} else {
/*
* The core will lose power & this VPE will not continue
* so it can simply halt here.
*/
MIPS: pm-cps: Add MIPSr6 CPU support This patch adds support for CPUs implementing the MIPSr6 ISA to the CPS power management code. Three changes are necessary: 1. In MIPSr6, coupled coherence is necessary when CPUS implement multiple Virtual Processors (VPs). 2. MIPSr6 virtual processors are more like real cores and cannot yield to other VPs on the same core, so drop the MT ASE yield instruction. 3. To halt a MIPSr6 VP, the CPC VP_STOP register is used rather than the MT ASE TCHalt CP0 register. Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com> Reviewed-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Kees Cook <keescook@chromium.org> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14225/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-07 17:45:17 +08:00
if (cpu_has_mipsmt) {
/* Halt the VPE via C0 tchalt register */
uasm_i_addiu(&p, t0, zero, TCHALT_H);
uasm_i_mtc0(&p, t0, 2, 4);
} else if (cpu_has_vp) {
/* Halt the VP via the CPC VP_STOP register */
unsigned int vpe_id;
vpe_id = cpu_vpe_id(&cpu_data[cpu]);
uasm_i_addiu(&p, t0, zero, 1 << vpe_id);
UASM_i_LA(&p, t1, (long)addr_cpc_cl_vp_stop());
uasm_i_sw(&p, t0, 0, t1);
} else {
BUG();
}
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
uasm_build_label(&l, p, lbl_secondary_hang);
uasm_il_b(&p, &r, lbl_secondary_hang);
uasm_i_nop(&p);
}
}
/*
* This is the point of no return - this VPE will now proceed to
* disable coherence. At this point we *must* be sure that no other
* VPE within the core will interfere with the L1 dcache.
*/
uasm_build_label(&l, p, lbl_disable_coherence);
/* Invalidate the L1 icache */
cps_gen_cache_routine(&p, &l, &r, &cpu_data[cpu].icache,
Index_Invalidate_I, lbl_invicache);
/* Writeback & invalidate the L1 dcache */
cps_gen_cache_routine(&p, &l, &r, &cpu_data[cpu].dcache,
Index_Writeback_Inv_D, lbl_flushdcache);
MIPS: pm-cps: Update comments on barrier instructions This code makes large use of barriers, which had quite vague descriptions. Update the comments to make the choice of barrier and reason for it more clear. Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com> Reviewed-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Kees Cook <keescook@chromium.org> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14220/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-07 17:45:12 +08:00
/* Barrier ensuring previous cache invalidates are complete */
MIPS: barrier: Add __SYNC() infrastructure Introduce an asm/sync.h header which provides infrastructure that can be used to generate sync instructions of various types, and for various reasons. For example if we need a sync instruction that provides a full completion barrier but only on systems which have weak memory ordering, we can generate the appropriate assembly code using: __SYNC(full, weak_ordering) When the kernel is configured to run on systems with weak memory ordering (ie. CONFIG_WEAK_ORDERING is selected) we'll emit a sync instruction. When the kernel is configured to run on systems with strong memory ordering (ie. CONFIG_WEAK_ORDERING is not selected) we'll emit nothing. The caller doesn't need to know which happened - it simply says what it needs & when, with no concern for checking the kernel configuration. There are some scenarios in which we may want to emit code only when we *didn't* emit a sync instruction. For example, some Loongson3 CPUs suffer from a bug that requires us to emit a sync instruction prior to each ll instruction (enabled by CONFIG_CPU_LOONGSON3_WORKAROUNDS). In cases where this bug workaround is enabled, it's wasteful to then have more generic code emit another sync instruction to provide barriers we need in general. A __SYNC_ELSE() macro allows for this, providing an extra argument that contains code to be assembled only in cases where the sync instruction was not emitted. For example if we have a scenario in which we generally want to emit a release barrier but for affected Loongson3 configurations upgrade that to a full completion barrier, we can do that like so: __SYNC_ELSE(full, loongson3_war, __SYNC(rl, always)) The assembly generated by these macros can be used either as inline assembly or in assembly source files. Differing types of sync as provided by MIPSr6 are defined, but currently they all generate a full completion barrier except in kernels configured for Cavium Octeon systems. There the wmb sync-type is used, and rmb syncs are omitted, as has been the case since commit 6b07d38aaa52 ("MIPS: Octeon: Use optimized memory barrier primitives."). Using __SYNC() with the wmb or rmb types will abstract away the Octeon specific behavior and allow us to later clean up asm/barrier.h code that currently includes a plethora of #ifdef's. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: linux-mips@vger.kernel.org Cc: Huacai Chen <chenhc@lemote.com> Cc: Jiaxun Yang <jiaxun.yang@flygoat.com> Cc: linux-kernel@vger.kernel.org
2019-10-02 05:53:07 +08:00
uasm_i_sync(&p, __SYNC_full);
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
uasm_i_ehb(&p);
MIPS: pm-cps: Support CM3 changes to Coherence Enable Register MIPS CM3 changed the management of coherence. Instead of a coherence control register with a bitmask of coherent domains, CM3 simply has a coherence enable register with a single bit to enable coherence of the local core. Support this by clearing and setting this single bit to disable / enable coherence. Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com> Reviewed-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Tony Wu <tung7970@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Nikolay Martynov <mar.kolya@gmail.com> Cc: Kees Cook <keescook@chromium.org> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14226/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-07 17:45:18 +08:00
if (mips_cm_revision() < CM_REV_CM3) {
/*
* Disable all but self interventions. The load from COHCTL is
* defined by the interAptiv & proAptiv SUMs as ensuring that the
* operation resulting from the preceding store is complete.
*/
MIPS: Abstract CPU core & VP(E) ID access through accessor functions We currently have fields in struct cpuinfo_mips for the core & VP(E) ID of a particular CPU, and various pieces of code directly access those fields. This patch abstracts such access by introducing accessor functions cpu_core(), cpu_set_core(), cpu_vpe_id() & cpu_set_vpe_id() and having code that needs to access these values call those functions rather than directly accessing the struct cpuinfo_mips fields. This prepares us for changes to the way in which those values are stored in later patches. The cpu_vpe_id() function is introduced even though we already had a cpu_vpe_id() macro for a couple of reasons: 1) It's more consistent with the core, and future cluster, accessors. 2) It ensures a sensible return type without explicit casts. 3) It's generally preferable to use functions rather than macros. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/17009/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2017-08-13 10:49:35 +08:00
uasm_i_addiu(&p, t0, zero, 1 << cpu_core(&cpu_data[cpu]));
MIPS: pm-cps: Support CM3 changes to Coherence Enable Register MIPS CM3 changed the management of coherence. Instead of a coherence control register with a bitmask of coherent domains, CM3 simply has a coherence enable register with a single bit to enable coherence of the local core. Support this by clearing and setting this single bit to disable / enable coherence. Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com> Reviewed-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Tony Wu <tung7970@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Nikolay Martynov <mar.kolya@gmail.com> Cc: Kees Cook <keescook@chromium.org> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14226/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-07 17:45:18 +08:00
uasm_i_sw(&p, t0, 0, r_pcohctl);
uasm_i_lw(&p, t0, 0, r_pcohctl);
/* Barrier to ensure write to coherence control is complete */
MIPS: barrier: Add __SYNC() infrastructure Introduce an asm/sync.h header which provides infrastructure that can be used to generate sync instructions of various types, and for various reasons. For example if we need a sync instruction that provides a full completion barrier but only on systems which have weak memory ordering, we can generate the appropriate assembly code using: __SYNC(full, weak_ordering) When the kernel is configured to run on systems with weak memory ordering (ie. CONFIG_WEAK_ORDERING is selected) we'll emit a sync instruction. When the kernel is configured to run on systems with strong memory ordering (ie. CONFIG_WEAK_ORDERING is not selected) we'll emit nothing. The caller doesn't need to know which happened - it simply says what it needs & when, with no concern for checking the kernel configuration. There are some scenarios in which we may want to emit code only when we *didn't* emit a sync instruction. For example, some Loongson3 CPUs suffer from a bug that requires us to emit a sync instruction prior to each ll instruction (enabled by CONFIG_CPU_LOONGSON3_WORKAROUNDS). In cases where this bug workaround is enabled, it's wasteful to then have more generic code emit another sync instruction to provide barriers we need in general. A __SYNC_ELSE() macro allows for this, providing an extra argument that contains code to be assembled only in cases where the sync instruction was not emitted. For example if we have a scenario in which we generally want to emit a release barrier but for affected Loongson3 configurations upgrade that to a full completion barrier, we can do that like so: __SYNC_ELSE(full, loongson3_war, __SYNC(rl, always)) The assembly generated by these macros can be used either as inline assembly or in assembly source files. Differing types of sync as provided by MIPSr6 are defined, but currently they all generate a full completion barrier except in kernels configured for Cavium Octeon systems. There the wmb sync-type is used, and rmb syncs are omitted, as has been the case since commit 6b07d38aaa52 ("MIPS: Octeon: Use optimized memory barrier primitives."). Using __SYNC() with the wmb or rmb types will abstract away the Octeon specific behavior and allow us to later clean up asm/barrier.h code that currently includes a plethora of #ifdef's. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: linux-mips@vger.kernel.org Cc: Huacai Chen <chenhc@lemote.com> Cc: Jiaxun Yang <jiaxun.yang@flygoat.com> Cc: linux-kernel@vger.kernel.org
2019-10-02 05:53:07 +08:00
uasm_i_sync(&p, __SYNC_full);
MIPS: pm-cps: Support CM3 changes to Coherence Enable Register MIPS CM3 changed the management of coherence. Instead of a coherence control register with a bitmask of coherent domains, CM3 simply has a coherence enable register with a single bit to enable coherence of the local core. Support this by clearing and setting this single bit to disable / enable coherence. Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com> Reviewed-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Tony Wu <tung7970@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Nikolay Martynov <mar.kolya@gmail.com> Cc: Kees Cook <keescook@chromium.org> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14226/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-07 17:45:18 +08:00
uasm_i_ehb(&p);
}
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
/* Disable coherence */
uasm_i_sw(&p, zero, 0, r_pcohctl);
uasm_i_lw(&p, t0, 0, r_pcohctl);
if (state >= CPS_PM_CLOCK_GATED) {
err = cps_gen_flush_fsb(&p, &l, &r, &cpu_data[cpu],
lbl_flush_fsb);
if (err)
goto out_err;
/* Determine the CPC command to issue */
switch (state) {
case CPS_PM_CLOCK_GATED:
cpc_cmd = CPC_Cx_CMD_CLOCKOFF;
break;
case CPS_PM_POWER_GATED:
cpc_cmd = CPC_Cx_CMD_PWRDOWN;
break;
default:
BUG();
goto out_err;
}
/* Issue the CPC command */
UASM_i_LA(&p, t0, (long)addr_cpc_cl_cmd());
uasm_i_addiu(&p, t1, zero, cpc_cmd);
uasm_i_sw(&p, t1, 0, t0);
if (state == CPS_PM_POWER_GATED) {
/* If anything goes wrong just hang */
uasm_build_label(&l, p, lbl_hang);
uasm_il_b(&p, &r, lbl_hang);
uasm_i_nop(&p);
/*
* There's no point generating more code, the core is
* powered down & if powered back up will run from the
* reset vector not from here.
*/
goto gen_done;
}
MIPS: pm-cps: Update comments on barrier instructions This code makes large use of barriers, which had quite vague descriptions. Update the comments to make the choice of barrier and reason for it more clear. Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com> Reviewed-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Kees Cook <keescook@chromium.org> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14220/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-07 17:45:12 +08:00
/* Barrier to ensure write to CPC command is complete */
MIPS: barrier: Add __SYNC() infrastructure Introduce an asm/sync.h header which provides infrastructure that can be used to generate sync instructions of various types, and for various reasons. For example if we need a sync instruction that provides a full completion barrier but only on systems which have weak memory ordering, we can generate the appropriate assembly code using: __SYNC(full, weak_ordering) When the kernel is configured to run on systems with weak memory ordering (ie. CONFIG_WEAK_ORDERING is selected) we'll emit a sync instruction. When the kernel is configured to run on systems with strong memory ordering (ie. CONFIG_WEAK_ORDERING is not selected) we'll emit nothing. The caller doesn't need to know which happened - it simply says what it needs & when, with no concern for checking the kernel configuration. There are some scenarios in which we may want to emit code only when we *didn't* emit a sync instruction. For example, some Loongson3 CPUs suffer from a bug that requires us to emit a sync instruction prior to each ll instruction (enabled by CONFIG_CPU_LOONGSON3_WORKAROUNDS). In cases where this bug workaround is enabled, it's wasteful to then have more generic code emit another sync instruction to provide barriers we need in general. A __SYNC_ELSE() macro allows for this, providing an extra argument that contains code to be assembled only in cases where the sync instruction was not emitted. For example if we have a scenario in which we generally want to emit a release barrier but for affected Loongson3 configurations upgrade that to a full completion barrier, we can do that like so: __SYNC_ELSE(full, loongson3_war, __SYNC(rl, always)) The assembly generated by these macros can be used either as inline assembly or in assembly source files. Differing types of sync as provided by MIPSr6 are defined, but currently they all generate a full completion barrier except in kernels configured for Cavium Octeon systems. There the wmb sync-type is used, and rmb syncs are omitted, as has been the case since commit 6b07d38aaa52 ("MIPS: Octeon: Use optimized memory barrier primitives."). Using __SYNC() with the wmb or rmb types will abstract away the Octeon specific behavior and allow us to later clean up asm/barrier.h code that currently includes a plethora of #ifdef's. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: linux-mips@vger.kernel.org Cc: Huacai Chen <chenhc@lemote.com> Cc: Jiaxun Yang <jiaxun.yang@flygoat.com> Cc: linux-kernel@vger.kernel.org
2019-10-02 05:53:07 +08:00
uasm_i_sync(&p, __SYNC_full);
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
uasm_i_ehb(&p);
}
if (state == CPS_PM_NC_WAIT) {
/*
* At this point it is safe for all VPEs to proceed with
* execution. This VPE will set the top bit of ready_count
* to indicate to the other VPEs that they may continue.
*/
if (coupled_coherence)
cps_gen_set_top_bit(&p, &l, &r, r_nc_count,
lbl_set_cont);
/*
* VPEs which did not disable coherence will continue
* executing, after coherence has been disabled, from this
* point.
*/
uasm_build_label(&l, p, lbl_secondary_cont);
/* Now perform our wait */
uasm_i_wait(&p, 0);
}
/*
* Re-enable coherence. Note that for CPS_PM_NC_WAIT all coupled VPEs
* will run this. The first will actually re-enable coherence & the
* rest will just be performing a rather unusual nop.
*/
MIPS: pm-cps: Support CM3 changes to Coherence Enable Register MIPS CM3 changed the management of coherence. Instead of a coherence control register with a bitmask of coherent domains, CM3 simply has a coherence enable register with a single bit to enable coherence of the local core. Support this by clearing and setting this single bit to disable / enable coherence. Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com> Reviewed-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Tony Wu <tung7970@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Nikolay Martynov <mar.kolya@gmail.com> Cc: Kees Cook <keescook@chromium.org> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14226/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-07 17:45:18 +08:00
uasm_i_addiu(&p, t0, zero, mips_cm_revision() < CM_REV_CM3
MIPS: CM: Use BIT/GENMASK for register fields, order & drop shifts There's no reason for us not to use BIT() & GENMASK() in asm/mips-cm.h when declaring macros corresponding to register fields. This patch modifies our definitions to do so. The *_SHF definitions are removed entirely - they duplicate information found in the masks, are infrequently used & can be replaced with use of __ffs() where needed. The *_MSK definitions then lose their _MSK suffix which is now somewhat redundant, and users are modified to match. The field definitions are moved to follow the appropriate register's accessor functions, which helps to keep the field definitions in order & to find the appropriate fields for a given register. Whilst here a comment is added describing each register & including its name, which is helpful both for linking the register back to hardware documentation & for grepping purposes. This also cleans up a couple of issues that became obvious as a result of making the changes described above: - We previously had definitions for GCR_Cx_RESET_EXT_BASE & a phony copy of that named GCR_RESET_EXT_BASE - a register which does not exist. The bad definitions were added by commit 497e803ebf98 ("MIPS: smp-cps: Ensure secondary cores start with EVA disabled") and made use of from boot_core(), which is now modified to use the GCR_Cx_RESET_EXT_BASE definitions. - We had a typo in CM_GCR_ERROR_CAUSE_ERRINGO_MSK - we now correctly define this as inFo rather than inGo. Now that we don't duplicate field information between _SHF & _MSK definitions, and keep the fields next to the register accessors, it will be much easier to spot & prevent any similar oddities being introduced in the future. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Acked-by: Thomas Gleixner <tglx@linutronix.de Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/17001/ Patchwork: https://patchwork.linux-mips.org/patch/17216/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2017-08-13 10:49:27 +08:00
? CM_GCR_Cx_COHERENCE_COHDOMAINEN
: CM3_GCR_Cx_COHERENCE_COHEN);
MIPS: pm-cps: Support CM3 changes to Coherence Enable Register MIPS CM3 changed the management of coherence. Instead of a coherence control register with a bitmask of coherent domains, CM3 simply has a coherence enable register with a single bit to enable coherence of the local core. Support this by clearing and setting this single bit to disable / enable coherence. Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com> Reviewed-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Tony Wu <tung7970@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Nikolay Martynov <mar.kolya@gmail.com> Cc: Kees Cook <keescook@chromium.org> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14226/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-07 17:45:18 +08:00
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
uasm_i_sw(&p, t0, 0, r_pcohctl);
uasm_i_lw(&p, t0, 0, r_pcohctl);
MIPS: pm-cps: Update comments on barrier instructions This code makes large use of barriers, which had quite vague descriptions. Update the comments to make the choice of barrier and reason for it more clear. Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com> Reviewed-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Kees Cook <keescook@chromium.org> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14220/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-07 17:45:12 +08:00
/* Barrier to ensure write to coherence control is complete */
MIPS: barrier: Add __SYNC() infrastructure Introduce an asm/sync.h header which provides infrastructure that can be used to generate sync instructions of various types, and for various reasons. For example if we need a sync instruction that provides a full completion barrier but only on systems which have weak memory ordering, we can generate the appropriate assembly code using: __SYNC(full, weak_ordering) When the kernel is configured to run on systems with weak memory ordering (ie. CONFIG_WEAK_ORDERING is selected) we'll emit a sync instruction. When the kernel is configured to run on systems with strong memory ordering (ie. CONFIG_WEAK_ORDERING is not selected) we'll emit nothing. The caller doesn't need to know which happened - it simply says what it needs & when, with no concern for checking the kernel configuration. There are some scenarios in which we may want to emit code only when we *didn't* emit a sync instruction. For example, some Loongson3 CPUs suffer from a bug that requires us to emit a sync instruction prior to each ll instruction (enabled by CONFIG_CPU_LOONGSON3_WORKAROUNDS). In cases where this bug workaround is enabled, it's wasteful to then have more generic code emit another sync instruction to provide barriers we need in general. A __SYNC_ELSE() macro allows for this, providing an extra argument that contains code to be assembled only in cases where the sync instruction was not emitted. For example if we have a scenario in which we generally want to emit a release barrier but for affected Loongson3 configurations upgrade that to a full completion barrier, we can do that like so: __SYNC_ELSE(full, loongson3_war, __SYNC(rl, always)) The assembly generated by these macros can be used either as inline assembly or in assembly source files. Differing types of sync as provided by MIPSr6 are defined, but currently they all generate a full completion barrier except in kernels configured for Cavium Octeon systems. There the wmb sync-type is used, and rmb syncs are omitted, as has been the case since commit 6b07d38aaa52 ("MIPS: Octeon: Use optimized memory barrier primitives."). Using __SYNC() with the wmb or rmb types will abstract away the Octeon specific behavior and allow us to later clean up asm/barrier.h code that currently includes a plethora of #ifdef's. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: linux-mips@vger.kernel.org Cc: Huacai Chen <chenhc@lemote.com> Cc: Jiaxun Yang <jiaxun.yang@flygoat.com> Cc: linux-kernel@vger.kernel.org
2019-10-02 05:53:07 +08:00
uasm_i_sync(&p, __SYNC_full);
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
uasm_i_ehb(&p);
if (coupled_coherence && (state == CPS_PM_NC_WAIT)) {
/* Decrement ready_count */
uasm_build_label(&l, p, lbl_decready);
MIPS: barrier: Add __SYNC() infrastructure Introduce an asm/sync.h header which provides infrastructure that can be used to generate sync instructions of various types, and for various reasons. For example if we need a sync instruction that provides a full completion barrier but only on systems which have weak memory ordering, we can generate the appropriate assembly code using: __SYNC(full, weak_ordering) When the kernel is configured to run on systems with weak memory ordering (ie. CONFIG_WEAK_ORDERING is selected) we'll emit a sync instruction. When the kernel is configured to run on systems with strong memory ordering (ie. CONFIG_WEAK_ORDERING is not selected) we'll emit nothing. The caller doesn't need to know which happened - it simply says what it needs & when, with no concern for checking the kernel configuration. There are some scenarios in which we may want to emit code only when we *didn't* emit a sync instruction. For example, some Loongson3 CPUs suffer from a bug that requires us to emit a sync instruction prior to each ll instruction (enabled by CONFIG_CPU_LOONGSON3_WORKAROUNDS). In cases where this bug workaround is enabled, it's wasteful to then have more generic code emit another sync instruction to provide barriers we need in general. A __SYNC_ELSE() macro allows for this, providing an extra argument that contains code to be assembled only in cases where the sync instruction was not emitted. For example if we have a scenario in which we generally want to emit a release barrier but for affected Loongson3 configurations upgrade that to a full completion barrier, we can do that like so: __SYNC_ELSE(full, loongson3_war, __SYNC(rl, always)) The assembly generated by these macros can be used either as inline assembly or in assembly source files. Differing types of sync as provided by MIPSr6 are defined, but currently they all generate a full completion barrier except in kernels configured for Cavium Octeon systems. There the wmb sync-type is used, and rmb syncs are omitted, as has been the case since commit 6b07d38aaa52 ("MIPS: Octeon: Use optimized memory barrier primitives."). Using __SYNC() with the wmb or rmb types will abstract away the Octeon specific behavior and allow us to later clean up asm/barrier.h code that currently includes a plethora of #ifdef's. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: linux-mips@vger.kernel.org Cc: Huacai Chen <chenhc@lemote.com> Cc: Jiaxun Yang <jiaxun.yang@flygoat.com> Cc: linux-kernel@vger.kernel.org
2019-10-02 05:53:07 +08:00
uasm_i_sync(&p, __SYNC_mb);
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
uasm_i_ll(&p, t1, 0, r_nc_count);
uasm_i_addiu(&p, t2, t1, -1);
uasm_i_sc(&p, t2, 0, r_nc_count);
uasm_il_beqz(&p, &r, t2, lbl_decready);
uasm_i_andi(&p, v0, t1, (1 << fls(smp_num_siblings)) - 1);
MIPS: pm-cps: Update comments on barrier instructions This code makes large use of barriers, which had quite vague descriptions. Update the comments to make the choice of barrier and reason for it more clear. Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com> Reviewed-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Kees Cook <keescook@chromium.org> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14220/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-07 17:45:12 +08:00
/* Barrier ensuring all CPUs see the updated r_nc_count value */
MIPS: barrier: Add __SYNC() infrastructure Introduce an asm/sync.h header which provides infrastructure that can be used to generate sync instructions of various types, and for various reasons. For example if we need a sync instruction that provides a full completion barrier but only on systems which have weak memory ordering, we can generate the appropriate assembly code using: __SYNC(full, weak_ordering) When the kernel is configured to run on systems with weak memory ordering (ie. CONFIG_WEAK_ORDERING is selected) we'll emit a sync instruction. When the kernel is configured to run on systems with strong memory ordering (ie. CONFIG_WEAK_ORDERING is not selected) we'll emit nothing. The caller doesn't need to know which happened - it simply says what it needs & when, with no concern for checking the kernel configuration. There are some scenarios in which we may want to emit code only when we *didn't* emit a sync instruction. For example, some Loongson3 CPUs suffer from a bug that requires us to emit a sync instruction prior to each ll instruction (enabled by CONFIG_CPU_LOONGSON3_WORKAROUNDS). In cases where this bug workaround is enabled, it's wasteful to then have more generic code emit another sync instruction to provide barriers we need in general. A __SYNC_ELSE() macro allows for this, providing an extra argument that contains code to be assembled only in cases where the sync instruction was not emitted. For example if we have a scenario in which we generally want to emit a release barrier but for affected Loongson3 configurations upgrade that to a full completion barrier, we can do that like so: __SYNC_ELSE(full, loongson3_war, __SYNC(rl, always)) The assembly generated by these macros can be used either as inline assembly or in assembly source files. Differing types of sync as provided by MIPSr6 are defined, but currently they all generate a full completion barrier except in kernels configured for Cavium Octeon systems. There the wmb sync-type is used, and rmb syncs are omitted, as has been the case since commit 6b07d38aaa52 ("MIPS: Octeon: Use optimized memory barrier primitives."). Using __SYNC() with the wmb or rmb types will abstract away the Octeon specific behavior and allow us to later clean up asm/barrier.h code that currently includes a plethora of #ifdef's. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: linux-mips@vger.kernel.org Cc: Huacai Chen <chenhc@lemote.com> Cc: Jiaxun Yang <jiaxun.yang@flygoat.com> Cc: linux-kernel@vger.kernel.org
2019-10-02 05:53:07 +08:00
uasm_i_sync(&p, __SYNC_mb);
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
}
if (coupled_coherence && (state == CPS_PM_CLOCK_GATED)) {
/*
* At this point it is safe for all VPEs to proceed with
* execution. This VPE will set the top bit of ready_count
* to indicate to the other VPEs that they may continue.
*/
cps_gen_set_top_bit(&p, &l, &r, r_nc_count, lbl_set_cont);
/*
* This core will be reliant upon another core sending a
* power-up command to the CPC in order to resume operation.
* Thus an arbitrary VPE can't trigger the core leaving the
* idle state and the one that disables coherence might as well
* be the one to re-enable it. The rest will continue from here
* after that has been done.
*/
uasm_build_label(&l, p, lbl_secondary_cont);
MIPS: pm-cps: Update comments on barrier instructions This code makes large use of barriers, which had quite vague descriptions. Update the comments to make the choice of barrier and reason for it more clear. Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com> Reviewed-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Kees Cook <keescook@chromium.org> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14220/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-07 17:45:12 +08:00
/* Barrier ensuring all CPUs see the updated r_nc_count value */
MIPS: barrier: Add __SYNC() infrastructure Introduce an asm/sync.h header which provides infrastructure that can be used to generate sync instructions of various types, and for various reasons. For example if we need a sync instruction that provides a full completion barrier but only on systems which have weak memory ordering, we can generate the appropriate assembly code using: __SYNC(full, weak_ordering) When the kernel is configured to run on systems with weak memory ordering (ie. CONFIG_WEAK_ORDERING is selected) we'll emit a sync instruction. When the kernel is configured to run on systems with strong memory ordering (ie. CONFIG_WEAK_ORDERING is not selected) we'll emit nothing. The caller doesn't need to know which happened - it simply says what it needs & when, with no concern for checking the kernel configuration. There are some scenarios in which we may want to emit code only when we *didn't* emit a sync instruction. For example, some Loongson3 CPUs suffer from a bug that requires us to emit a sync instruction prior to each ll instruction (enabled by CONFIG_CPU_LOONGSON3_WORKAROUNDS). In cases where this bug workaround is enabled, it's wasteful to then have more generic code emit another sync instruction to provide barriers we need in general. A __SYNC_ELSE() macro allows for this, providing an extra argument that contains code to be assembled only in cases where the sync instruction was not emitted. For example if we have a scenario in which we generally want to emit a release barrier but for affected Loongson3 configurations upgrade that to a full completion barrier, we can do that like so: __SYNC_ELSE(full, loongson3_war, __SYNC(rl, always)) The assembly generated by these macros can be used either as inline assembly or in assembly source files. Differing types of sync as provided by MIPSr6 are defined, but currently they all generate a full completion barrier except in kernels configured for Cavium Octeon systems. There the wmb sync-type is used, and rmb syncs are omitted, as has been the case since commit 6b07d38aaa52 ("MIPS: Octeon: Use optimized memory barrier primitives."). Using __SYNC() with the wmb or rmb types will abstract away the Octeon specific behavior and allow us to later clean up asm/barrier.h code that currently includes a plethora of #ifdef's. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: linux-mips@vger.kernel.org Cc: Huacai Chen <chenhc@lemote.com> Cc: Jiaxun Yang <jiaxun.yang@flygoat.com> Cc: linux-kernel@vger.kernel.org
2019-10-02 05:53:07 +08:00
uasm_i_sync(&p, __SYNC_mb);
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
}
/* The core is coherent, time to return to C code */
uasm_i_jr(&p, ra);
uasm_i_nop(&p);
gen_done:
/* Ensure the code didn't exceed the resources allocated for it */
BUG_ON((p - buf) > max_instrs);
BUG_ON((l - labels) > ARRAY_SIZE(labels));
BUG_ON((r - relocs) > ARRAY_SIZE(relocs));
/* Patch branch offsets */
uasm_resolve_relocs(relocs, labels);
/* Flush the icache */
local_flush_icache_range((unsigned long)buf, (unsigned long)p);
return buf;
out_err:
kfree(buf);
return NULL;
}
MIPS: pm-cps: Generate idle state entry code when CPUs are onlined The MIPS Coherent Processing System (CPS) power management code has previously generated code used to enter low power idle states once during boot for all CPUs. This has the drawback that if a CPU is present in the system but not being used (for example due to the maxcpus kernel parameter) then we encounter problems due to not having probed that CPU for information about its type & properties. The result of this is that we generate entry code which is both unused, potentially entirely invalid & likely to be unsuitable for the CPU in question anyway. Avoid this by generating idle state entry code only when a CPU is brought online. This way we only ever generate code for CPUs that we know we've probed the properties of, and that will actually be used. [ralf@linux-mips.org: Resolve merge conflict.] Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14259/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-14 18:00:27 +08:00
static int cps_pm_online_cpu(unsigned int cpu)
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
{
enum cps_pm_state state;
MIPS: Abstract CPU core & VP(E) ID access through accessor functions We currently have fields in struct cpuinfo_mips for the core & VP(E) ID of a particular CPU, and various pieces of code directly access those fields. This patch abstracts such access by introducing accessor functions cpu_core(), cpu_set_core(), cpu_vpe_id() & cpu_set_vpe_id() and having code that needs to access these values call those functions rather than directly accessing the struct cpuinfo_mips fields. This prepares us for changes to the way in which those values are stored in later patches. The cpu_vpe_id() function is introduced even though we already had a cpu_vpe_id() macro for a couple of reasons: 1) It's more consistent with the core, and future cluster, accessors. 2) It ensures a sensible return type without explicit casts. 3) It's generally preferable to use functions rather than macros. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/17009/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2017-08-13 10:49:35 +08:00
unsigned core = cpu_core(&cpu_data[cpu]);
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
void *entry_fn, *core_rc;
for (state = CPS_PM_NC_WAIT; state < CPS_PM_STATE_COUNT; state++) {
if (per_cpu(nc_asm_enter, core)[state])
continue;
if (!test_bit(state, state_support))
continue;
entry_fn = cps_gen_entry_code(cpu, state);
if (!entry_fn) {
pr_err("Failed to generate core %u state %u entry\n",
core, state);
clear_bit(state, state_support);
}
per_cpu(nc_asm_enter, core)[state] = entry_fn;
}
if (!per_cpu(ready_count, core)) {
MIPS: pm-cps: Drop manual cache-line alignment of ready_count We allocate memory for a ready_count variable per-CPU, which is accessed via a cached non-coherent TLB mapping to perform synchronisation between threads within the core using LL/SC instructions. In order to ensure that the variable is contained within its own data cache line we allocate 2 lines worth of memory & align the resulting pointer to a line boundary. This is however unnecessary, since kmalloc is guaranteed to return memory which is at least cache-line aligned (see ARCH_DMA_MINALIGN). Stop the redundant manual alignment. Besides cleaning up the code & avoiding needless work, this has the side effect of avoiding an arithmetic error found by Bryan on 64 bit systems due to the 32 bit size of the former dlinesz. This led the ready_count variable to have its upper 32b cleared erroneously for MIPS64 kernels, causing problems when ready_count was later used on MIPS64 via cpuidle. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Fixes: 3179d37ee1ed ("MIPS: pm-cps: add PM state entry code for CPS systems") Reported-by: Bryan O'Donoghue <bryan.odonoghue@imgtec.com> Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@imgtec.com> Tested-by: Bryan O'Donoghue <bryan.odonoghue@imgtec.com> Cc: linux-mips@linux-mips.org Cc: stable <stable@vger.kernel.org> # v3.16+ Patchwork: https://patchwork.linux-mips.org/patch/15383/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2017-03-03 06:02:40 +08:00
core_rc = kmalloc(sizeof(u32), GFP_KERNEL);
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
if (!core_rc) {
pr_err("Failed allocate core %u ready_count\n", core);
return -ENOMEM;
}
per_cpu(ready_count, core) = core_rc;
}
return 0;
}
MIPS: pm-cps: Block system suspend when a JTAG probe is present If a JTAG probe is connected to a MIPS cluster, then the CPC detects it and latches the CPC.STAT_CONF.EJTAG_PROBE bit to 1. While set, attempting to send a power-down command to a core will be blocked, and the CPC will instead send the core to clock-off state. This can interfere with systems fully entering a low power state where all cores, CM, GIC, etc are powered down. Detect that a JTAG probe is / has been connected to the cluster and block the suspend attempt. Attempting to suspend the system while a JTAG probe is connected now yields: # echo mem > /sys/power/state [ 11.654000] PM: Syncing filesystems ... done. [ 11.658000] JTAG probe is connected - abort suspend -sh: echo: write error: Operation not permitted # To restore suspend, the JTAG probe should be disconnected or put into quiescent state. Platform code can then clear the CPC.STAT_CONF.EJTAG_PROBE bit. Reported-by: Ed Blake <ed.blake@sondrel.com> Signed-off-by: Matt Redfearn <matt.redfearn@mips.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Paul Burton <paul.burton@mips.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/18641/ Signed-off-by: James Hogan <jhogan@kernel.org>
2018-02-20 17:58:16 +08:00
static int cps_pm_power_notifier(struct notifier_block *this,
unsigned long event, void *ptr)
{
unsigned int stat;
switch (event) {
case PM_SUSPEND_PREPARE:
stat = read_cpc_cl_stat_conf();
/*
* If we're attempting to suspend the system and power down all
* of the cores, the JTAG detect bit indicates that the CPC will
* instead put the cores into clock-off state. In this state
* a connected debugger can cause the CPU to attempt
* interactions with the powered down system. At best this will
* fail. At worst, it can hang the NoC, requiring a hard reset.
* To avoid this, just block system suspend if a JTAG probe
* is detected.
*/
if (stat & CPC_Cx_STAT_CONF_EJTAG_PROBE) {
pr_warn("JTAG probe is connected - abort suspend\n");
return NOTIFY_BAD;
}
return NOTIFY_DONE;
default:
return NOTIFY_DONE;
}
}
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
static int __init cps_pm_init(void)
{
/* A CM is required for all non-coherent states */
if (!mips_cm_present()) {
pr_warn("pm-cps: no CM, non-coherent states unavailable\n");
MIPS: pm-cps: Generate idle state entry code when CPUs are onlined The MIPS Coherent Processing System (CPS) power management code has previously generated code used to enter low power idle states once during boot for all CPUs. This has the drawback that if a CPU is present in the system but not being used (for example due to the maxcpus kernel parameter) then we encounter problems due to not having probed that CPU for information about its type & properties. The result of this is that we generate entry code which is both unused, potentially entirely invalid & likely to be unsuitable for the CPU in question anyway. Avoid this by generating idle state entry code only when a CPU is brought online. This way we only ever generate code for CPUs that we know we've probed the properties of, and that will actually be used. [ralf@linux-mips.org: Resolve merge conflict.] Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14259/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-14 18:00:27 +08:00
return 0;
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
}
/*
* If interrupts were enabled whilst running a wait instruction on a
* non-coherent core then the VPE may end up processing interrupts
* whilst non-coherent. That would be bad.
*/
if (cpu_wait == r4k_wait_irqoff)
set_bit(CPS_PM_NC_WAIT, state_support);
else
pr_warn("pm-cps: non-coherent wait unavailable\n");
/* Detect whether a CPC is present */
if (mips_cpc_present()) {
/* Detect whether clock gating is implemented */
MIPS: CPC: Use BIT/GENMASK for register fields, order & drop shifts Tidy up asm/mips-cpc.h in a similar way to what "MIPS: CM: Use BIT/GENMASK for register fields, order & drop shifts" did for asm/mips-cm.h. We use BIT() & GENMASK() to simplify the definition of register fields, drop the _SHF definitions since that information can be found in the _MSK ones, and then drop the _MSK suffix. Fields definitions are moved to be next to the appropriate register definition, making it easier to link the two & keep everything ordered by register address. Comments are added including the name of each register & a brief description of its purpose which helps to understand what registers are for, link them back to hardware documentation or grep for them. Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/17003/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2017-08-13 10:49:29 +08:00
if (read_cpc_cl_stat_conf() & CPC_Cx_STAT_CONF_CLKGAT_IMPL)
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
set_bit(CPS_PM_CLOCK_GATED, state_support);
else
pr_warn("pm-cps: CPC does not support clock gating\n");
/* Power gating is available with CPS SMP & any CPC */
if (mips_cps_smp_in_use())
set_bit(CPS_PM_POWER_GATED, state_support);
else
pr_warn("pm-cps: CPS SMP not in use, power gating unavailable\n");
} else {
pr_warn("pm-cps: no CPC, clock & power gating unavailable\n");
}
MIPS: pm-cps: Block system suspend when a JTAG probe is present If a JTAG probe is connected to a MIPS cluster, then the CPC detects it and latches the CPC.STAT_CONF.EJTAG_PROBE bit to 1. While set, attempting to send a power-down command to a core will be blocked, and the CPC will instead send the core to clock-off state. This can interfere with systems fully entering a low power state where all cores, CM, GIC, etc are powered down. Detect that a JTAG probe is / has been connected to the cluster and block the suspend attempt. Attempting to suspend the system while a JTAG probe is connected now yields: # echo mem > /sys/power/state [ 11.654000] PM: Syncing filesystems ... done. [ 11.658000] JTAG probe is connected - abort suspend -sh: echo: write error: Operation not permitted # To restore suspend, the JTAG probe should be disconnected or put into quiescent state. Platform code can then clear the CPC.STAT_CONF.EJTAG_PROBE bit. Reported-by: Ed Blake <ed.blake@sondrel.com> Signed-off-by: Matt Redfearn <matt.redfearn@mips.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Paul Burton <paul.burton@mips.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/18641/ Signed-off-by: James Hogan <jhogan@kernel.org>
2018-02-20 17:58:16 +08:00
pm_notifier(cps_pm_power_notifier, 0);
cpu/hotplug: Cleanup state names When the state names got added a script was used to add the extra argument to the calls. The script basically converted the state constant to a string, but the cleanup to convert these strings into meaningful ones did not happen. Replace all the useless strings with 'subsys/xxx/yyy:state' strings which are used in all the other places already. Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Sebastian Siewior <bigeasy@linutronix.de> Link: http://lkml.kernel.org/r/20161221192112.085444152@linutronix.de Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2016-12-22 03:19:54 +08:00
return cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "mips/cps_pm:online",
MIPS: pm-cps: Generate idle state entry code when CPUs are onlined The MIPS Coherent Processing System (CPS) power management code has previously generated code used to enter low power idle states once during boot for all CPUs. This has the drawback that if a CPU is present in the system but not being used (for example due to the maxcpus kernel parameter) then we encounter problems due to not having probed that CPU for information about its type & properties. The result of this is that we generate entry code which is both unused, potentially entirely invalid & likely to be unsuitable for the CPU in question anyway. Avoid this by generating idle state entry code only when a CPU is brought online. This way we only ever generate code for CPUs that we know we've probed the properties of, and that will actually be used. [ralf@linux-mips.org: Resolve merge conflict.] Signed-off-by: Paul Burton <paul.burton@imgtec.com> Cc: Adam Buchbinder <adam.buchbinder@gmail.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/14259/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2016-09-14 18:00:27 +08:00
cps_pm_online_cpu, NULL);
MIPS: pm-cps: add PM state entry code for CPS systems This patch adds code to generate entry & exit code for various low power states available on systems based around the MIPS Coherent Processing System architecture (ie. those with a Coherence Manager, Global Interrupt Controller & for >=CM2 a Cluster Power Controller). States supported are: - Non-coherent wait. This state first leaves the coherent domain and then executes a regular MIPS wait instruction. Power savings are found from the elimination of coherency interventions between the core and any other coherent requestors in the system. - Clock gated. This state leaves the coherent domain and then gates the clock input to the core. This removes all dynamic power from the core but leaves the core at the mercy of another to restart its clock. Register state is preserved, but the core can not service interrupts whilst its clock is gated. - Power gated. This deepest state removes all power input to the core. All register state is lost and the core will restart execution from its BEV when another core powers it back up. Because register state is lost this state requires cooperation with the CONFIG_MIPS_CPS SMP implementation in order for the core to exit the state successfully. The code will detect which states are available on the current system during boot & generate the entry/exit code for those states. This will be used by cpuidle & hotplug implementations. Signed-off-by: Paul Burton <paul.burton@imgtec.com>
2014-04-14 18:00:56 +08:00
}
arch_initcall(cps_pm_init);
Copy Permalink