------------------------------------------------------------------------------
Supported Distro list and supported package list
------------------------------------------------------------------------------

################################################################################
SuSE:
################################################################################

x86_64
	SLES 12 SP1
		Gold (3.12.49-11)		(default, xen)
	SLES 12 SP2
		Gold (4.4.21-69)		(default)
	SLES 12 SP3
		Gold (4.4.73-5)			(default)
	SLES 15
		Gold (4.12.14-23)		(default)

################################################################################
Red Hat, Cent OS, Oracle Enterprise Linux
################################################################################

i686

	RHEL6/OEL6/CENTOS6
		Update 8 (2.6.32-642.el6.i686)		(el6)
		Update 9 (2.6.32-696.el6.i686)		(el6)
		Update 10 (2.6.32-754.el6.i686)		(el6)

x86_64

	RHEL6/OEL6/CENTOS6
		Update 8 (2.6.32-642.el6.x86_64)	(el6)
		Update 9 (2.6.32-696.el6.x86_64)	(el6)
		Update 10 (2.6.32-754.el6.x86_64)	(el6)

	RHEL7/OEL7/CENTOS7
		Update3 (3.10.0-514.el7.x86_64)		(el7)
		Update4 (3.10.0-693.el7.x86_64)		(el7)
		Update5 (3.10.0-862.el7.x86_64)		(el7)	


################################################################################
UEK supported by Oracle.
################################################################################

i386
	OEL6
		Update 9UEK (2.6.39-400.294.3)		(el6uek)
		Update 10UEK (2.6.39-400.299.3)		(el6uek)

x86_64
	OEL6
		Update 9UEK (4.1.12-61.1.28)			(el6uek)
		Update 10UEK (4.1.12-124.16.4)			(el6uek)
	OEL7
		Update 4UEK (4.1.12-94.3.9)		(el7uek)
		Update 5UEK (4.1.12-112.16.4)		(el7uek)

################################################################################
Ubuntu:
################################################################################

i686
	Ubuntu16.04LTS (4.4.0-21-generic)

x86_64
	Ubuntu16.04LTS (4.4.0-21-generic)
	Ubuntu18.04LTS (4.15.0-20-generic)


################################################################################
Citrix:
################################################################################

NOTE:
	Only RPM support is provided for Citrix's XenServer 7.x OSes.
	Citrix has changed the DUD (Driver Update Disk) process in XenServer 7.x
	and made it mandatory for ISO images to be signed with GPG keys and
	the GPG public key must be present in the base OS installation media.
	Due to this DUD support is dropped for XenServer 7.x.
	Also, XenServer 7.x OS by default installs out-of-box megaraid_sas driver
	rpm while installing the OS. After installing the OS, user can upgrade to
	the latest Citrix XenServer 7.x megaraid_sas driver rpm provided as below,

# rpm -Uvh avago-megaraid-sas-07.703.01.00_CitrixXen7.1-1.x86_64.rpm

Note regarding MegaRAID driver RPM support for XenServer 7.3 and 7.4:
	Citrix Xenserver 7.2 MegaRAID driver RPM itself supports Citrix XenServer 7.3 and 7.4.
	Since kernel version in XenServer 7.3 and 7.4 is the same as XenServer 7.2 (4.4.0+10),
	MegaRAID driver RPM avago-megaraid-sas CitrixXen 7.2 will work on XenServer 7.3 and 7.4.

x86_64
	Citrix7
		Update 1 LTSR	(4.4.0+2)
		Update 2 CR	(4.4.0+10)
		Update 3 CR	(4.4.0+10)
		Update 4 CR	(4.4.0+10)
		

Few additional notes:
=====================
1. There is a bug in anaconda installer of RHEL7.2 which causes driver provided via DUD not getting loaded, if any of
   controllers supported by inbox MR driver is present at setup. In this case, inbox driver will be loaded during OS installation.
   To load DUD driver, there is a workaround-  boot the installation kernel with following option:

   modprobe.blacklist=megaraid_sas

   This will cause the in-box driver not to be loaded automatically, but will not prevent the DUD package to load the modules later.


2. For few distros(e.g.fedora23), while building binary RPM from source RPM, debuginfo package building is mandatory
   but MR build scripts do not have debugfiles.list required to build MR driver's debuginfo RPM. Not having debugfiles.list
   can cause failure to build binary RPM from source RPM. On such distros, while creating binary RPM from source RPM(using rpmbuild)
   use below options-
   #rpmbuild -ba --define "debug_package %{nil}"  file_name.spec

3. SLES12 SP3 inbox driver has a code bug that results in driver updates via DUD to fail. The issue manifests itself only when inbox driver is unloaded
   as part of DUD installation. OS installations can still continue using inbox driver which supports latest generation MegaRAID controllers. 
   Customers can then upgrade to latest out-of-box driver using RPM method.
   Please refer below BugZilla for more details.
   https://bugzilla.novell.com/show_bug.cgi?id=1053681”
 
Recommendation - 
1.	DUD for SLES12SP3 is not really required since inbox driver supports upto Ventura series controllers.  
2.	Install SLES12SP3 OS using default inbox megaraid_sas driver. 
3.	Post installation user can pick kmp rpm, souce rpm and upgrade the driver. 
Note - rmmod and manual insmod on live system is not possible due above mentioned issue.


4. Many Linux kernel distribution fails >4MB contiguous memory allocation request on x86 platform. Most of the linux distribution on x86 platform supports
   PAGE_SIZE=4K and max possible physical contiguous memory supported is 4MB (2^10 * PAGE_SIZE). This upper limit (4MB) depends upon architecture and kernel
   configuration (like PAGE_SIZE etc.).
 
   In MegaRaid JBOD personality which supports adapter QD  = ~9k, memory required for IO request frame pool is >4 MB.
   <megaraid_sas> driver will work in reduced QD mode (QD set to 8172) so that corresponding memory requirement for IO request frames is  <= 4MB.
   Below is git commit id of upstream patch which makes driver is reduced QD mode due to >4MB memory can not be allocated-
  
   e97e673 scsi: megaraid_sas: Retry with reduced queue depth when alloc fails for higher QD 
  
 
   In newer kernels, CMA (contiguous memory allocator) feature can be used to allocate large physical contiguous memory. 
   CMA can be enabled by adding cma=xxxx in grub entry of kernel. This will reserve some memory reserved for larger memory allocations on boot time. 
   Once grub file is edited, reboot the system and check for CmaTotal in output of cat /proc/meminfo it should reflect memory reserved
   in grub config file.
   e.g. If cma=16M is set in grub config file(menu.lst/grub.cfg), then CmaTotal should be set as below- 
  # cat /proc/meminfo 
  --
  --
  CmaTotal:          16384 kB
  CmaFree:               0 kB
  --
 
  For more details on CMA refer to this link- https://lwn.net/Articles/486301/

5. List of megaraid_sas driver binaries with retpoline support-

   1. Driver binaries for below OS versions have retpoline support built using "thunk-extern" GCC option
      as the GA kernels itself have retpoline support enabled:
       - RHEL7.5 GA and any future RHEL7.x major releases that are supported
       - RHEL6.10 GA and any future RHEL6.x major releases that are supported
       - SLES15 GA and any future SLES15 Service Pack releases that are supported
       - OEL7.5 UEK GA and any future OEL7.x UEK releases that are supported
       - OEL6.10 UEK GA and any future OEL6.x UEK releases that are supported
       - Ubuntu 18.04 LTS and any future Ubuntu LTS releases that are supported

   2. Driver binaries for below OS versions have retpoline support built using "thunk-inline" GCC option:
       - RHEL7.4 GA
       - RHEL7.3 GA
       - RHEL6.9 GA
       - RHEL6.8 GA
       - SLES12 SP3
       - SLES12 SP2
       - SLES12 SP1
       - OEL7.4 UEK GA
       - OEL6.9 UEK GA

   In output of "modinfo megaraid_sas.ko", "retpoline:   Y" indicates compiled driver module is retpoline compliant.

6. Some information on driver binaries built for OS GA kernels which does not have retpoline support:
    1. For those OS where GA kernel does not have retpoline support, if OSV has released updated GCC
       package, in that case driver binaries will be built using "thunk-inline" GCC option.
       Refer above list for OS GA versions where retpoline support is currently available.
    2. In this case, both driver RPMs and DUDs are compiled with GCC option:
       "-mindirect-branch=thunk-inline -mindirect-branch-register"
       They are safe from Spectre v2 vulnerabilities.
    3. "modinfo megaraid_sas.ko" will show "retpoline :Y" for driver module built with retpoline support. 
    4. Retpoline kernels have "CONFIG_RETPOLINE=y" in kernel config file.
    5. Driver binaries (both DUD and RPM) will work for both OS GA as well as retpoline kernels.

7. On Oracle Linux 6.10, after driver RPM is installed and subsequent reboot, inbox driver will still be loaded.
	After driver RPM installation a new initramfs is not getting created containing new driver module.

	During RPM installation, below errors will be seen:
	gzip: /boot/initramfs-4.1.12-124.16.4.el6uek.x86_64.img: not in gzip format
	
	gzip: /boot/initramfs-4.1.12-124.16.4.el6uek.x86_64.tmp: not in gzip format
	
	This is a known issue in one of the OS installer scripts "weak-modules" in Oracle Linux 6.10.

	Refer below BugZilla filed with OS vendor for more details:
	https://bugzilla.oracle.com/bugzilla/show_bug.cgi?id=16497

	As a workaround, after RPM installation completes regenerate new initramfs manually using below commands:
	1. first make a backup of the existing initramfs
	# cp /boot/initramfs-4.1.12-124.16.4.el6uek.x86_64.img /boot/initramfs-4.1.12-124.16.4.el6uek.x86_64.img.back

	2. Create a new initramfs using dracut:
	# dracut -f

8. On RHEL 6.9/OEL 6.9 and later, OS installation behind MegaRAID controller in JBOD personality mode does not work
	As mentioned in Note 4 above, MegaRAID JBOD personality supports large queue depth and driver needs to allocate DMA memory > 4MB for the request pools. 

	On RHEL 6.9 and above, allocation of > 4MB DMA memory using dma_pool_alloc with GFP_KERNEL flag hangs forever due to a kernel bug.
	Refer BugZilla: https://bugzilla.redhat.com/show_bug.cgi?id=1481956
	
	This issue is specific to RHEL 6.x OSes that are based off 2.6.32 kernels.

	For the above kernel bug, a workaround fix was provided in driver but this is not present in inbox driver of RHEL 6.9 and later.

	During OS installation, inbox driver load will hang indefinitely and we will not be able to load OOB driver using DUD as well.
	
	As a workaround, during OS installation provide below module parameter to disable RDPQ mode.
	This will reduce FW queue depth to 1k and DMA memory allocation will succeed.

	Provide boot option on the kernel command line during installation: "megaraid_sas.rdpq_enable=0"

9. On SLES12 SP2, if first controller has a faulty firmware, after reboot the other valid controllers are not detected.
	When there are multiple controllers connected, PCI probe is called synchronously.
	If the first controller has a faulty firmware, driver waits 180seconds for FW to come to ready state before failing the probe.
	On SLES12 SP2, systemd-udevd that is responsible for driver probing has a very low timeout of 30seconds during boot. 
	Sicne first controller probe takes 180seconds to fail, systemd-udevd timeout would have elapsed.
	And probe for remaining controllers are killed. Due to this remaining valid controllers are also not configured.
	Refer bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1058028

	Below workaround was suggested by OS vendor to enable asynchronous probing in driver for all controllers.
	1. In /etc/modprobe.d/ directory, create a file named megaraid_sas.conf.
	2. Add below line to enable async probing:
	options megaraid_sas async_probe
	
	OR
	
	1. By enabling below kernel command line parameter in grub:
	'megaraid_sas.async_probe'


