Easy to Create, Easy to Change - Easy to use!


Troubleshooting Network Slowness - SMB and Oplocks again

Started by DataEase
You will need to Sign In to be able to comment on the Blog!

Troubleshooting Network Slowness - SMB and Oplocks again

  • 1. Introduction


    The Server Message Block (SMB) Protocol is a network file sharing protocol, and as implemented in Microsoft Windows is known as Microsoft SMB Protocol. The set of message packets that defines a particular version of the protocol is called a dialect. The Common Internet File System (CIFS) Protocol is a dialect of SMB. Both SMB and CIFS are also available on VMS, several versions of Unix, and other operating systems.

    There has been a lot of white/blue/green and yellow papers produced on Registry Settings etc. to help with network slowness. We have tested them all and some but the only thing we have learnt is that you can't tweak yourself out of these problems. 

    As a general rule one must assume that a newer version of WIndows Server is slower than the last due to "better" security and more functionality. This is as always to be compensated by the assumption of better hardware that will make the situation liveable.

    So if you move from a "successful" 2003 environment to a 2008R2 environment on the same network/server you will see slower performance.

    If you are really lucky you will reach some border levels and your entire setup will go into a tailspin and slow down dramatically.

    What you need to do then is not to try to tweak your network by attacking the registry settings and find wonder cures on the internet. There aren't any the same way you can't lose weight without eating slower and walking faster.

    A quick fix is better network cabling, better switches, better routers and a better server....

    But as with loosing weight its not the hardware that loose you the weight but you, so when you experience performance problems while moving environment take this as a good omen and use it to improve the performance of your application.

    In our experience 99.99% of performance problems can be vastly reduced or entirely removed by fixing the design flaws in your application being it badly designed forms or procedures.

    We have seen procedures that went from taking 3 hours on 2003 to 6 hours on 2008R2 and hence no longer was viable be reduced to a mere couple of seconds while properly designed.

    However it is nothing wrong in understanding why your server goes slower the newer it is so please read on below:


    Microsoft SMB Protocol and CIFS Protocol Overview

    http://msdn.microsoft.com/en-us/library/windows/desktop/aa365233(v=vs.85).aspx


    Server Message Block overview

    http://technet.microsoft.com/en-us/library/hh831795.aspx


    1.1SMB Versions and Negotiated Versions

    There are several different versions of SMB used by Windows operating systems:


    • CIFS – The ancient version of SMB that was part of Microsoft Windows NT 4.0 in 1996. SMB1 supersedes this version.
    • SMB 1.0 (or SMB1) – The version used in Windows 2000, Windows XP, Windows Server 2003 and Windows Server 2003 R2
    • SMB 2.0 (technically SMB2 version 2.002) – The version used in Windows Vista (SP1 or later) and Windows Server 2008 (or any SP)
    • SMB 2.1 ((technically SMB2 version 2.1) – The version used in Windows 7 (or any SP) and Windows Server 2008 R2 (or any SP)
    • SMB 3.0 (or SMB3) – The version used in Windows 8 and Windows Server 2012
    • SMB 3.02 (or SMB3) – The version used in Windows 8.1 and Windows Server 2012 R2

    Windows NT is no longer supported, so CIFS is definitely out. Windows Server 2003 R2 with a current service pack is under Extended Support, so SMB1 is still around for a little while. SMB 2.x in Windows Server 2008 and Windows Server 2008 R2 are under Mainstream Support until 2015. You can find the most current information on the support lifecycle page for Windows Server. The information is subject to the Microsoft Policy Disclaimer and Change Notice. You can use the support pages to also find support policy information for Windows XP, Windows Vista, Windows 7 and Windows 8.


    In Windows 8.1 and Windows Server 2012 R2, we introduced the option to completely disable CIFS/SMB1 support, including the actual removal of the related binaries. While this is not the default configuration, we recommend disabling this older version of the protocol in scenarios where it’s not useful, like Hyper-V over SMB. You can find details about this new option in item 7 of this blog post:What’s new in SMB PowerShell in Windows Server 2012 R2.


    Negotiated Versions


    Here’s a table to help you understand what version you will end up using, depending on what Windows version is running as the SMB client and what version of Windows is running as the SMB server:

    OSWindows 8.1 WS 2012 R2Windows 8 WS 2012Windows 7 WS 2008 R2Windows Vista WS 2008Previous versions
    Windows 8.1 WS 2012 R2SMB 3.02SMB 3.0SMB 2.1SMB 2.0SMB 1.0
    Windows 8 WS 2012SMB 3.0SMB 3.0SMB 2.1SMB 2.0SMB 1.0
    Windows 7 WS 2008 R2SMB 2.1SMB 2.1SMB 2.1SMB 2.0SMB 1.0
    Windows Vista WS 2008SMB 2.0SMB 2.0SMB 2.0SMB 2.0SMB 1.0
    Previous versionsSMB 1.0SMB 1.0SMB 1.0SMB 1.0SMB 1.0

    * WS = Windows Server

    1.2 Check, Enable and Disable SMB Versions in Windows operating systems

    In Windows 8 or Windows Server 2012 and later, there is a new PowerShell cmdlet that can easily tell you what version of SMB the client has negotiated with the File Server. You simply access a remote file server (or create a new mapping to it) and use Get-SmbConnection.

    To enable and disable SMBv1, SMBv2, and SMBv3 in Windows Vista, Windows Server 2008, Windows 7, Windows Server 2008 R2, Windows 8, and Windows Server 2012, please follow the steps in the article below.Warning: We do not recommend that you disable SMBv2 or SMBv3. Disable SMBv2 or SMBv3 only as a temporary troubleshooting measure. Do not leave SMBv2 or SMBv3 disabled.


    http://support.microsoft.com/kb/2696547


    1.3 Features and Capabilities

    Here’s a very short summary of what changed with each version of SMB:

    • From SMB 1.0 to SMB 2.0 - The first major redesign of SMB
    • Increased file sharing scalability
    • Improved performance
      • Request compounding
      • Asynchronous operations
      • Larger reads/writes
    • More secure and robust
      • Small command set
      • Signing now uses HMAC SHA-256 instead of MD5
      • SMB2 durability
    • From SMB 2.0 to SMB 2.1
      • File leasing improvements
      • Large MTU support
      • BranchCache
    • From SMB 2.1 to SMB 3.0
      • Availability
        • SMB Transparent Failover
        • SMB Witness
        • SMB Multichannel
      • Performance
        • SMB Scale-Out
        • SMB Direct (SMB 3.0 over RDMA)
        • SMB Multichannel
        • Directory Leasing
        • BranchCache V2
      • Backup
        • VSS for Remote File Shares
      • Security
        • SMB Encryption using AES-CCM (Optional)
        • Signing now uses AES-CMAC
      • Management
        • SMB PowerShell
        • Improved Performance Counters
        • Improved Eventing
    • From SMB 3.0 to SMB 3.02
      • Automatic rebalancing of Scale-Out File Server clients
      • Improved performance of SMB Direct (SMB over RDMA)
      • Support for multiple SMB instances on a Scale-Out File Server

    You can get additional details on the SMB 2.0 improvements listed above athttp://blogs.technet.com/b/josebda/archive/2008/12/09/smb2-a-complete-redesign-of-the-main-remote-file-protocol-for-windows.aspx

    You can get additional details on the SMB 3.0 improvements listed above athttp://blogs.technet.com/b/josebda/archive/2012/05/03/updated-links-on-windows-server-2012-file-server-and-smb-3-0.aspx

    You can get additional details on the SMB 3.02 improvements in Windows Server 2012 R2 athttp://technet.microsoft.com/en-us/library/hh831474.aspx

    1.4 Related Registry Keys

    • HKLM\SYSTEM\CurrentControlSet\Services\MrxSmb\Parameters\
    • DeferredOpensEnabled – Indicates whether the Redirector can defer opens for certain cases where the file does not really need to be opened, such as for certain delete requests and adjusting file attributes. This defaults to true and is stored in the Redirector variable MRxSmbDeferredOpensEnabled.
    • OplocksDisabled – Whether the Redirector should not request oplocks, this defaults to false (the Redirector will request oplocks) and is stored in the variable MrxSmbOplocksDisabled.
    • CscEnabled – Whether Client Side Caching is enabled. This value defaults to true and stored in MRxSmbIsCscEnabled. It is used to determine whether to execute CSC operations when called. If CSC is enabled, several other parameters controlling CSC behavior are checked, such as CscEnabledDCON, CscEnableTransitionByDefault, and CscEnableAutoDial. CSC will be discussed in depth in its own module, so will be only mentioned in this module when it is necessary to understanding the operation of the Redirector.
    • DisableShadowLoopback – Whether to disable the behavior of the Redirector getting a handle to loopback opens (opens on the same machine) so that it can shortcut the network path to the resource and just access local files locally. Shadow opens are enabled by default, and this registry value can be used to turn them off. It is stored in the global Redirector variable RxSmbDisableShadowLoopback.
    • IgnoreBindingOrder – Controls whether the Redirector should use the binding order specified in the registry and controlled by the Network Connections UI, or ignore this order when choosing a transport provider to provide a connection to the server. By default the Redirector will ignore the binding order and can use any transport. The results of this setting are stored in the variable MRxSmbObeyBindingOrder.
    • HKLM\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters\
    • Security Signature settings – The RequireSecuritySignature setting is stored in MRxSmbSecuritySignaturesRequired, EnableSecuritySignature in MRxSmbSecuritySignaturesEnabled, RequireExtendedSignature in MRxSmbExtendedSignaturesRequired, and EnableExtendedSignature in MRxSmbExtendedSignaturesEnabled. Note that the Extended Security Signatures assume the regular security signatures are enabled, so those settings are adjusted if necessary based on the extended settings. If extended signatures are required, regular signatures have to be required.
    • EnablePlainTextPassword – Support for using plain text passwords can be turned on using this key. They are disabled by default.
    • OffLineFileTimeoutIntervalInSeconds – Used to set the expiration time for timing out an Exchange (discussed later) when the exchange is accessing an offline file. This value defaults to 1000 seconds, but can be changed in the registry and is stored in the global Redirector variable OffLineFileTimeoutInterval
    • SessTimeout – This is the amount of time the client waits for the server to respond to an outstanding request. The default value is 60 seconds (Windows Vista). When the client does not receive the response to a request before the Request Expiration Timer expires, it will reset the connection because the operation is considered blocked. In Windows 8, the request expiration timer for the SMB 2 Negotiate is set to a smaller value, typically under 20 seconds, so that if a node of a continuously available (CA) cluster server is not responding, the SMB 3.0 client can expedite failover to the other node.
    • ExtendedSessTimeout – Stored in the ExtendedSessTimeoutInterval variable, this value is used to extend the timeout on exchanges for servers that require an extended session timeout as listed in the ServersWithExtendedSessTimeout key. These are third party servers that handle SMB sessions with different processes and vary dramatically on the time required to process SMB requests. The default value is 1000 seconds. If the client is running at least Windows 7 and ExtendedSessTimeout is not configured (By Default), the timeout is extended to four times the value of SessTimeout (4 * SessTimeout).
    • MaxNumOfExchangesForPipelineReadWrite – This value is used to determine the maximum number of write exchanges that can be pipelined to a server. The default is 8 and the value is stored in the variable MaxNumOfExchangesForPipelineReadWrite.
    • Win9xSessionRestriction – This value defaults to false, but is used to impose a restriction on Windows 9x clients that they can only have one active non-NULL session with the server at a time. Also, existing session based connections (VNETROOTS) are scavenged immediately, without a timeout to allow them to be reused.
    • EnableCachingOnWriteOnlyOpens – This value can cause the Redirector to attempt to open a file that is being opened for write only access in a manner that will enable the Redirector to cache the file data. If the open fails, the request will revert back to the original requested access. The value of this parameter defaults to false and is stored in the MRxSmbEnableCachingOnWriteOnlyOpens variable.
    • DisableByteRangeLockingOnReadOnlyFiles – This parameter defaults to false, but if set to true will cause level II oplocks to automatically be upgraded to batch oplocks on read-only files opened for read only access. It is stored in the variable DisableByteRangeLockingOnReadOnlyFiles.
    • EnableDownLevelLogOff – False by default, this value controls whether a Logoff SMB will be sent to down-level servers when a session is being closed. If this is false, and the server has not negotiated to the NT SMB dialect or does not support NT Status codes, the logoff will not be sent because we aren’t sure that server will understand the request. The value is stored in MrxSmbEnableDownLevelLogOff.
    • HKLM\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters\
    • ResilientTimeout – This timer is started when the transport connection associated with a resilient handle is lost. It controls the amount of time the server keeps a resilient handle active after the transport connection to the client is lost. The default value is 300 seconds (Windows 7, Server 2008 R2, 8, Server 2012).
    • DurableHandleV2TimeoutInSecond – This timer is started when the transport connection associated with a durable handle is lost. It controls the amount of time the server keeps a durable handle active after the transport connection to the client is lost. The default value is 60 seconds (Windows 8, Windows Server 2012). The maximum value is 300 seconds.
    • HKLM\SYSTEM\CurrentControlSet\Services\SMBWitness\Parameters\
    • KeepAliveInterval – This functionality was introduced for SMB 3.0 in Windows 8 and Windows Server 2012. The witness protocol is used to explicitly notify a client of resource changes that have occurred on a highly available cluster server. This enables faster recovery from unplanned failures, so that the client does not need to wait for TCP timeouts. The default value is 20 minutes (Windows 8, Windows Server 2012).
    • HKLM\System\CurrentControlSet\Services\SmbDirect\Parameters\
    • ConnectTimeoutInMs – Establish a connection and complete negotiation. ConnectTimeoutInMs is the deadline for the remote peer to accept the connection request and complete SMB Direct negotiation. Default is 120 seconds (Windows 8).
    • AcceptTimeoutInMs – Accept negotiation: The SMB Direct Negotiate request should be received before AcceptTimeoutInMs expires. The servers starts this timer as soon as it accepted the connection. Default is 5 seconds (Windows 8).
    • IdleConnectionTimeoutInMs – This timer is per-connection. It is the amount of time the connection can be idle without receiving a message from the remote peer. Before the local peer terminates the connection, it sends a keep alive request to the remote peer and applies a keep alive timer. Default is Default: 120 seconds (Windows 8).
    • KeepaliveResponseTimeoutInMs – This attribute is per-connection. It defines the timeout to wait for the peer response for a keep-alive message on an idle RDMA connection. Default is 5 seconds (Windows 8).
    • CreditGrantTimeoutInMs – This timer is per-connection. It regulates the amount of time that the local peer waits for the remote peer to grant Send credits before disconnecting the connection. This timer is started when the local peer runs out of Send credits. Default is 5 seconds (Windows 8).

Written by DataEase 06/11/18 at 15:27:49 Dataease [{8}]FIVE