Nginx and Memcached, a 400% boost!

2010-07-23  来源:本站原创  分类:Internet  人气:215 

If Web architectures, performance, or scalability are topics you would like to keep on top of (who does not!), Then chances are, you've heard of Nginx ("Engine x"). Originally developed by Igor Sysoev for rambler . ru (Second largest Russian Web-site), it is a high-performance HTTP Server / Reverse proxy known for its stability , performance , and ease of use . The great track record, a lot of great modules, and an active development community have rightfully earned it a Steady uptick of users, and most recently, a notable mention in the Netcraft report .

Memcached module - an easy 4x speed multiplier

Memcached, the darling of every web-developer, is capable of turning almost any application into a speed-demon. Benchmarking one of my own Rails applications resulted in ~ 850 req / s on commodity, non-optimized hardware - more than enough in the case of this Application. However, what IF we took out of the equation Mongrel? Nginx, by default, comes prepackaged with the Memcached module , which allows U.S. to bypass the Mongrel Servers and talk to Memcached directly. Same hardware, and a Quick test later: ~ 3,550 req / s, or almost a 400% improvement! Not bad for a five minute tweak!

Nginx and Memcached, a 400% boost!

Think smart, forget cache invalidations

The only SnAg in our scheme for Easy performance gains comes with the fact that more often than not, our Application Servers contain additional caching policies (read invalidations / Authentication), and MIME type Logic. The former, as recently documented by Tobias Lütke and Geoffrey Grosenbach , IF properly thought through can be solved with some clever URL rewriting policies and Automatic TTL timeouts. When implemented correctly, we could simply set the memcached Key to be the full Request URL, allowing U.S. to completely bypass our App. Servers.

MIME-type logic

MIME type magic can be as easy as complex as we wish. If you only serve one content type ('text / html', for example), the solution is simple:

> Nginx-default.conf

location /dynamic_request {
   # Set default type to text/html
   default_type  text/html;

   # ...
}

Dynamic argument types, just for fun

However, if we want to serve multiple content-types, or perhaps even parameterize the request type in a query string, we've got some extra work to do. Not unlike any other HTTP server, Nginx checks the filetype extension at the end of every request path to determine the correct content-type header, a solution which unfortunately breaks down in majority of modern, URL friendly web-applications:

1. GET / dynamic_request.js - Content-Type = text / javascript
2. GET / dynamic_request - Content-Type =?
3. GET / dynamic_request? Format = js - Content-Type =?

Case 1 is easily solved by Nginx directly. Case 2 is tricky, but can be solved via a 'default_type' line in the config as document above. And case 3 will require some additional logic - namely, we can hardcode a rule to rewrite our dynamic query string parameters to automagically add an extension to the path of each incoming request:

> Nginx-rewrite.conf

location /dynamic_request {
           # append an extenstion for proper MIME type detection
           if ($args ~* format=json) { rewrite ^/dynamic_request/?(.*)$ /dynamic_request.js$1 break; }
           if ($args ~* format=xml)  { rewrite ^/dynamic_request/?(.*)$ /dynamic_request.xml$1 break; }

           memcached_pass 127.0.0.1:11211;
           error_page 404 = @dynamic_request;
}


nginx.conf (Full Nginx sample config)

Downloads: 9549 File Size: 1.4 KB

That should do the trick! Cache invalidations are handled, MIME types are served correctly, and our app. Servers are bypassed in 95% + of the cases. Instead, Nginx talks directly to Memcached and only proxies the cache misses - an easy 400% performance boost!

相关文章
  • Nginx and Memcached, a 400% boost! 2010-07-23

    If Web architectures, performance, or scalability are topics you would like to keep on top of (who does not!), Then chances are, you've heard of Nginx ("Engine x"). Originally developed by Igor Sysoev for rambler . ru (Second largest Russian Web

  • nginx + tomcat + memcached server load balancing configuration notes 2011-09-04

    How to install nginx, tomcat6 memcached installation please refer to http://wudx.iteye.com/blog/1165260 memcached-session-manager configuration Let tomcat called memcached to store the session early on is a very mature solution, the source of the msm

  • CentOS 5.4下的Memcache安装步骤(Linux+Nginx+PHP+Memcached) 2013-12-24

    CentOS 5.4下的Memcache安装步骤分享,想要配置Linux+Nginx+PHP+Memcached运行环境的朋友可以参考下 一.源码包准备 服务器端主要是安装memcache服务器端,目前的最新版本是 memcached-v1.4.4 . 下载:http://memcached.googlecode.com/files/memcached-1.4.4.tar.gz 另外,Memcache用到了libevent这个库用于Socket的处理,所以还需要安装libevent, libev

  • 企业网站架构之Nginx+tomcat+memcached集群 2014-08-11

    nginx+tomcat+memcached应用 系统环境:RHEL6.4 x64 iptables -F and selinux is disabled 主机角色:node1 :192.168.0.24 :lnmp环境 tomcat memcached node2 :192.168.0.99 : tomcat memcached 软件下载:在lnmp环境上测试tomcat,使用nginx简单发布jsp jdk-6u32-linux-x64.bin apache-tomcat-7.0.37.ta

  • Nginx Tomcat Memcached cluster structures 2011-05-20

    1. Ngnix Download http://nginx.org/en/download.html This example uses the 1.0.2 version of nginx 2. Tomcat Download http://tomcat.apache.org This example uses the 6.0.18 version of tomcat 3. Memcached downloads http://memcached.org This example uses

  • Nginx 日志产生大量400错误日志 2012-10-22

    最近一个打点的日志,发现nginx日志记录里多了很多400错误,没有访问地址. 221.239.20.160 - - [18/Oct/2012:20:00:02 +0800] "-" 400 0 "-" "-" "-" 现象大致: 网页访问的时候马上产生一条正常的访问日志,过个10来s产生400错误日志. 监控浏览器请求在访问网页后没有其他请求,再用网络抓包也没抓到请求记录. 但是这条日志的ip又是本机的 大致原因(nginx

  • Linux下nginx+tomcat+memcached集群 2013-04-24

    集群分为横向集群和纵向集群(纵向意思就是在一台服务器上 横向就是多台服务器 就这么个意思) 一.软件版本 nginx-0.7.65 apache-tomcat-6.0.26 jdk-6u20-linux-i586-rpm.bin memcached-1.4.0.tar.gz 二.结构图 三.内容服务器 所有的代码都存放在内容服务器上,内容服务器提供文件共享,集群机通过cifs方式映射为web目录,这样便于统一管理和即时同步. 集群机对内容的共享详见三.3. 四.集群机的配置 1.Java环境安装

  • nginx+tomcat +memcached 2014-03-31

    <TI> <T2> . \ / . . X . . / \ . <M1> <M2> Tomcat-1 (T1) 将 session 存储在 memcached-2 (T2)上.只有当 M2 不可用时,T1 才将 session 存 储在 memcached-1 上(M1 是 T1 failoverNode).使用这种配置的好处是,当 T1 和 M1 同时崩 溃时也不会丢失 session 会话,避免单点故障 我们需要准备两个节点(nginx1,nginx2)

  • Nginx+Tomcat+Memcached集群Session共享(windows) 2015-01-23

    环境: windows 7 + nginx-1.1.5 + memcached-1.2.6-win32-bin + apache-tomcat-7.0.20-windows-x86 + jdk-6u6-windows-i586-p(JDK 1.6 , memcached-1.2.6-win32-bin 下载地址:http://code.jellycan.com/files/memcached-1.2.6-win32-bin.zip . 需要用到的 jar 包: memcached-2.6.jar

  • centos 部署php+nginx+memadmin + memcached 2014-07-24

    nginx http://nginx.org/download/nginx-1.7.3.tar.gz php http://cn2.php.net/distributions/php-5.5.14.tar.gz php.memcache插件 http://pecl.php.net/get/memcache-3.0.8.tgz memadmin http://www.junopen.com/memadmin/memadmin-1.0.12.tar.gz install nginx tar -zxf

  • Nginx学习之八:Nginx开发者学习资源攻略 2014-07-04

    开发者资源 Getting Started with Nginx Frequently Asked Questions Nginx Official Documentation (Non-Wiki) CodingStyle SourceTricks 英文指南 Evan Miller's Guide to Nginx Module Development Evan Miller's Advanced Topics In Nginx Module Development Slides of agen

  • 高性能Web服务器Nginx的配置与部署研究(13)应用模块之Memcached模块+Proxy_Cache双层缓存模式 2012-03-20

    本文来自:CSDN博客专栏<Nginx高性能Web服务器> 及Poechant技术博客,转载请注明出处. 通过<高性能Web服务器Nginx的配置与部署研究--(11)应用模块之Memcached模块的两大应用场景>一文,我们知道Nginx从Memcached读取数据的方式,如果命中,那么效率是相当高的.那么: (转载请注明来自Poechant的CSDN博客:http://blog.csdn.net/poechant/article/details/7179891) 1. 如果不命

  • 高性能Web服务器Nginx的配置与部署研究(12)应用模块之Memcached做文件缓存时压缩引起的问题 2012-03-20

    本文来自:CSDN博客专栏<Nginx高性能Web服务器> 及Poechant技术博客,转载请注明出处. 在上一篇文章中,提到的Nginx的Memcached模块应用场景,主要是作为文件缓存.然后就发现了一个问题,当以字节数组方式缓存较大的文件时,缓存数据会被压缩,从而在读取的时候出现问题. (本文欢迎转载,尊重他人劳动,转载请注明出处:http://blog.csdn.net/poechant/article/details/7177603) 解决方法很简单,就是在MemcachedClie

  • 高性能Web服务器Nginx的配置与部署研究(11)应用模块之Memcached模块的两大应用场景 2012-03-20

    本文来自:CSDN博客专栏<Nginx高性能Web服务器> 及Poechant技术博客,转载请注明出处. 一.应用场景1 (转载请注明:http://blog.csdn.net/poechant/article/details/7176921) 最近在一个项目中,用到了Nginx的Memcached模块,所以就在这个系列教程中提前把Memcached模块拿出来写了.另外发现最近我的博客文章频频被很多用采集器的网站拿走,帮我发扬光大,都不听我说声谢谢.在此还是希望我的博文被转载的时候能够被注明出

  • vanish(squid) + HAProxy + nginx + memcached(redis) 2013-12-10

    1. Load-balance then cache +-- Cache server #1 (varnish) -- App server #1 / Load Balancer (haproxy)-+---- Cache server #2 (varnish) -- App server #2 \ +-- Cache server #3 (varnish) -- App server #3 2. Cache then load-balance +-- App server #1 / Cache

  • nginx cache cache of five kinds of programs 2010-03-18

    Nginx seemingly never written a cache function, would only not sharing, it is not good habit ah. 1, one of the traditional cache (404) This approach is to nginx 404 error directed to the back end, then back to return to the page proxy_store save. Con

  • nginx 作为反向代理实现负载均衡的例子 2014-05-04

    以下我们就来举例说明如何使用 nginx 实现负载均衡.因为nginx在处理并发方面的优势,现在这个应用非常常见. nginx 这个轻量级.高性能的 web server 主要可以干两件事情: 〉直接作为http server(代替apache,对PHP需要FastCGI处理器支持): 〉另外一个功能就是作为反向代理服务器实现负载均衡 以下我们就来举例说明如何使用 nginx 实现负载均衡.因为nginx在处理并发方面的优势,现在这个应用非常常见.当然了Apache的 mod_proxy和mod

  • nginx的一些配置 2012-05-07

    一'Nginx加密访问页面(https) 1'cd /usr/local/nginx/conf 2'vim nginx.conf server { listen 443; server_name localhost; ssl on; ssl_certificate cert.pem; ssl_certificate_key cert.pem; ssl_session_timeout 5m; ssl_protocols SSLv2 SSLv3 TLSv1; ssl_ciphers HIGH:!aN

  • nginx做为反向代理实现负载均衡的例子 2012-10-06

    因为nginx在处理并发方面的优势,现在这个应用非常常见.当然了Apache的mod_proxy和mod_cache结合使用也可以实现对多台app server的反向代理和负载均衡,但是在并发处理方面apache还是没有nginx擅长. nginx作为反向代理实现负载均衡的例子: 转载请注明原文链接:http://blog.csdn.net/omohe/archive/2009/07/09/4335765.aspx 版本:v1.0 作者:OMO 最后修改时间:2009.07.09 1)环境: a

  • Boost::Bind 源码实现简化版 2012-11-21

    本着看懂boost::bind整体流程的角度出发,将boost::bind的源码精简贴出,去除了所有多余部分,只在说明问题,错误在所难免,希望大家勿喷. bind_t对象就是boost::bind方法的返回值,它保存了F和L的值.其中F为函数指针类型,L为listN类型(N的范围为0-9),它将保存用户调用boost::bind方法时传入的绑定参数.例如: void fun_1(int left, int right){//dosomething} boost::bind(&fun_1, 100