Use 64-bit Win Server 2008 servers for Web servers
Next version of SharePoint will be 64-bit only is a good enough reason to use a 64-bit operating system (Windows Server 2008) for all new installation. If you plan for and implement 64-bit MOSS now, it may make your upgrade path in the future a little easier.
The 64-bit systems offer direct access to more virtual and physical memory than 32-bit systems and process more data per clock cycle, enabling more scalable, higher performing computing solutions.
The /3GB switch is not recommended for SharePoint Server running on 32-bit OS. Microsoft recommends against using the /3 GB switch because most SharePoint site traffic involves sending large amounts of data through the operating system. Therefore, leaving only 1 GB of address space for the operating system can destabilize the computer.
Check out the "Benefits of 64-bit Windows" whitepaper from Microsoft to know more advantages of using 64-bits Servers.
Avoid mixing 32-bit and 64-bit servers
One should avoid to mix to 32-bit and 64-bit servers until you have any significant reason to do so. You can still mix and match 32 and 64 bit servers, but you must keep the same architecture at each tier of the farm. For example all front end (WFE) SharePoint servers must be either 32-bit or 64-bit. All index/search servers must be one or the other. All database servers (if you have more than 1 in a cluster for example) must be either 32-bit or 64-bit. But at each tier you can change architecture. For example you could have 2 front ends at 64-bit, 2 search index servers at 32-bit, and 2 database servers 32-bit. As long as each tier is the same architecture.
Mixing 32-bit and 64-bit servers can affect load balancing. However, there is a risk that the 32-bit servers might become overloaded if the network load balancer is configured to use a less-intelligent model such as round robin.
Additionally, deploying both 32-bit and 64-bit servers increases the maintenance overhead for the farm. This is because third-party applications, custom solutions, patches, and software updates for both architectures must be tracked and managed independently.
Limit the size of content database
Microsoft recommends that content database size should not be more than 100 GB to achieve optimal performance. In case your design requires a database larger than 100 GB, you should split content from a site collection that is approaching 100 GB into a new site collection in a separate content database to avoid performance or manageability issues.
Put SQL Server 2005 on a dedicated server
As a best practice use install SQL Server 2005 on a dedicated server that is not running any other farm roles, unless you are deploying your system on a stand-alone server. You should use 64-bit version of SQL Server 2005 on a 64-bit operating system, unless you have a significant business reason not to.
Turn Off the features you don’t need
SharePoint offers many features but you don’t necessary need all for your application. Resources will be more efficiently used if you only turn on the features relevant to your application.
As with any software, some features take up more resources than others. Especially the features that rely on the Microsoft SharePoint Timer service, such as alerts and usage analysis log processing, can have more of an impact on your server performance than other features. Different features can affect server resources for different reasons, but usually performance is impacted when a feature must be run on demand when a user performs a particular action.
Use Kerberos authentication
NTLM and Kerberos are the two popular authentication mechanisms for SharePoint.
NTLM Authentication is the well-known and uses challenge-response for authentication. NTLM is easy and does not require any special configuration. As Microsoft likes to say, “It just works.”
Kerberos, on the other hand, is a more complex ticket-based authentication mechanism that authenticates the client to the server and authenticates the server to the client. While Kerberos is more secure with several other advantages though it can be a bit challenging to configure it properly.
Microsoft recommends using Kerberos authentication for farms with heavy usage as it can return authentication request results quickly. Kerberos caches information about the client after authentication and therefore it requires fewer round trips to the domain controllers in comparison to NTLM. Once a client gets a service ticket for a service from the Key Distribution Center (KDC -Domain controller on a windows network) it can reuse that ticket to authenticate with the service without having to go back to the domain controller for a new ticket. NTLM requires trips to the domain controller on every authentication request. This means that it can perform better than NTLM particularly in large farm environments.
SharePoint Joel blog post on Kerberos authentication
Configure Kerberos authentication (Office SharePoint Server) on TechNet