Hibernate cache configuration / batch processing

2010-04-07  来源:本站原创  分类:Java  人气:596 

Hibernate cache configuration / batch processing Keywords: hibernate
Hibernate In addition to automatically conduct the affairs of a Session-level cache, the second-level cache will need to achieve org.hibernate.cache.CacheProvider interface, Hibernate has achieved a number of caching, developers can configure the use, at the same time enable the second-level cache, Configure hibernate.cache.use_second_level_cache is true.

Optional values:

org.hibernate.cache.HashtableCacheProvide
org.hibernate.cache.EhCacheProvider
org.hibernate.cache.OSCacheProvider
org.hibernate.cache.SwarmCacheProvider
org.hibernate.cache.TreeCacheProvider ...

For example:
(1). Hibernate.cfg.xml
Xml Code
<property name="hibernate.cache.provider_class"> org.hibernate.cache.EhCacheProvider </ property>
<property name="hibernate.cache.use_second_level_cache"> true </ property>

<property name="hibernate.cache.provider_class"> org.hibernate.cache.EhCacheProvider </ property>
<property name="hibernate.cache.use_second_level_cache"> true </ property>

(2) spring
Xml Code
<prop key="hibernate.cache.provider_class"> org.hibernate.cache.EhCacheProvider </ prop>
<prop key="hibernate.cache.use_second_level_cache"> true </ prop>

<prop key="hibernate.cache.provider_class"> org.hibernate.cache.EhCacheProvider </ prop>
<prop key="hibernate.cache.use_second_level_cache"> true </ prop>

By default, Hibernate will not have all the entities in the object cache, so what we need to specify the cache object, the object in the physical mapping file (the corresponding <class> label internal), add the following configuration:
<cache usage="read-only"/> or hibernate.cfg.xml add the following configuration:
<class-cache usage="read-only"/>

usage = "read-only" is "read-only" cache strategy.

Note that this <cache> tag can be placed only within <class> label, and must be at the front of <id> label.

Hibernate second-level cache is a SessionFactory-level cache, which allows multiple Session between the shared use of the cache need to use third-party components, a new version of Hibernate would be EHcache 2 cache as the default implementation.

Cache synchronization strategy: Cache synchronization strategy determines the data objects in the cache access rules, we must specify the appropriate class for each entity cache synchronization strategy. Hibernate provides 4 different cache synchronization strategy:
1.read-only: read-only. Those who will never change the data may be used.
2.nonstrict-read-write: If the program is under the concurrent access to the data synchronization request is not strict, and the data update less frequently, using the cache synchronization strategies available to better performance.
3.read-write: a strict read and write caching. Based on the time stamp to determine the mechanism to achieve the "read committed" isolation level transaction. Used for data synchronization requirements, but does not support distributed cache used in the practical application of the most cache synchronization strategy.
4.transactional: transactional cache, you must run in the JTA transaction environment. In this cache, the cache-related operations are added to the transaction (this is similar to a cache memory database), such as transaction fails, then the buffer pool data together roll back to the transaction prior to the start state. transactional cache achieved a "Repeatable read" isolation level transaction, effectively guarantee the legitimacy of the data to meet the critical data in the cache, Hibernate built-in cache, only to support transactional JBossCache cache.

A variety of cache implementation

hibernate.cache.use_minimal_puts: Does optimized to minimize the secondary cache read and write operation, when the cache cluster optimization. Optional values: true (default): Enable read and write operations to minimize. false: disable minimize to read and write operations.
hibernate.cache.use_query_cache: Does the cache query results (query terms to use). Optional values: true: Caching query results. false: do not cache query results
hibernate.cache.use_second_level_cache: No second-level cache is enabled. Optional values: true: Enable second-level cache. false: do not use second-level cache.
hibernate.cache.query_cache_factory: Set a custom query cache type the full name, the cache class must implement the org.hibernate.cache.QueryCache interface. Optional values: (1) org.hibernate.cache.StandardQueryCache (default). (2) a custom cache implementation class.
hibernate.cache.region_prefix: 2 cache prefix name.
hibernate.cache.use_structured_entries: whether to use a structured way of cached objects. Optional values: true: in a structured way cached objects. false: do not use a structured way to cache the object.

Attached: echcache.xml
Xml Code
<? xml version = "1.0" encoding = "UTF-8"?>
<ehcache>
<diskStore path="java.io.tmpdir"/>
<defaultCache
maxElementsInMemory = "10000"
eternal = "false"
timeToIdleSeconds = "120"
timeToLiveSeconds = "120"
overflowToDisk = "true" />

<cache name = "org.hibernate.cache.StandardQueryCache"
maxElementsInMemory = "10000"
eternal = "false"
timeToIdleSeconds = "200"
timeToLiveSeconds = "500"
overflowToDisk = "true" />

<cache name = "org.hibernate.cache.UpdateTimestampsCache"
maxElementsInMemory = "5000"
eternal = "true"
overflowToDisk = "true" />

</ ehcache>

<? xml version = "1.0" encoding = "UTF-8"?>
<ehcache>
<diskStore path="java.io.tmpdir"/>
<defaultCache
maxElementsInMemory = "10000"
eternal = "false"
timeToIdleSeconds = "120"
timeToLiveSeconds = "120"
overflowToDisk = "true" />

<cache name = "org.hibernate.cache.StandardQueryCache"
maxElementsInMemory = "10000"
eternal = "false"
timeToIdleSeconds = "200"
timeToLiveSeconds = "500"
overflowToDisk = "true" />

<cache name = "org.hibernate.cache.UpdateTimestampsCache"
maxElementsInMemory = "5000"
eternal = "true"
overflowToDisk = "true" />

</ ehcache>

maxElementsInMemory attribute is used to specify the cache can hold up to the number of objects.
eternal attribute specifies the cache is permanent.
timeToIdleSeconds attribute specifies how long the cache is idle will be cleared away unused.
timeToLiveSeconds attribute specifies the length of the life of the cache.
diskPersistent property specifies whether the cache is persisted to the hard drive, save the path designated by the <diskStore> tag.

Test, log4j.properties
Java code
log4j.logger.org.hibernate = warn
log4j.logger.org.hibernate.cache = debug

log4j.logger.org.hibernate = warn
log4j.logger.org.hibernate.cache = debug

Batch processing:
Because these two kinds of Hibernate have different cache management mechanism for the two cache, we can size it related configuration, while for the internal cache, Hibernate have adopted a "laissez-faire" attitude, and its capacity and There is no limit. Crux of the problem is now found, we do mass data inserted when generating so many objects will be included in the internal cache (internal cache is to do in memory cache), so your system memory will be 1.1 points will be eroded, if the Pushing the final system, "bombing", and also reasonable that the.

We think about how better to deal with this problem? Some development conditions must also use Hibernate to deal with, of course, some projects are more flexible, you can go to seek other methods.

Here are two recommended methods:
(1): Optimizing Hibernate, procedures and timely clear the cache using sub-insertion method.
(2): bypass the Hibernate API, directly through the JDBC API to do bulk insert, this method is the best performance, but also the fastest.
For the above in Method 1, the basic idea is as follows: optimization of Hibernate, in the configuration file to set hibernate.jdbc.batch_size parameter to specify the number of SQL for each submission; procedures and timely use of sub-paragraph insert the Clear Cache method (Session to achieve the asynchronous write-behind, which allows Hibernate to explicitly write the batch), which is inserted every time a certain amount of data in them removed from the internal cache, the release of occupied memory.
Set hibernate.jdbc.batch_size parameters, refer to the following configuration.
Xml Code
<hibernate-configuration>
<session-factory>
.........
<property name=" hibernate.jdbc.batch_size"> 50 </ property>
.........
<session-factory>
<hibernate-configuration>

<hibernate-configuration>
<session-factory>
.........
<property name=" hibernate.jdbc.batch_size"> 50 </ property>
.........
<session-factory>
<hibernate-configuration>

Configuration parameters hibernate.jdbc.batch_size reason for this is as little as possible reading of the database, hibernate.jdbc.batch_size larger parameter values, read the fewer the number of databases, the faster. As can be seen from the above configuration, Hibernate is to wait until the program has accumulated to 50 to submit SQL batch after.

hibernate.jdbc.batch_size parameter values may not be set too the bigger the better, from a performance standpoint is also open to question. This must consider the actual situation, as appropriate, set up, in general is set to meet the needs of 30,50.
Java code
Session session = HibernateUtil.currentSession ();
Transatcion tx = session.beginTransaction ();
for (int i = 0; i <10000; i + +)
... (
Student st = new Student ();
st.setName ( "feifei");
session.save (st);
if (i% 50 == 0) / / for every 50 data as a processing unit
... (
session.flush (); / / maintain data synchronization with the database
session.clear (); / / clear the internal cache all the data, timely release the memory occupied by
)
)
tx.commit ();
.........

Session session = HibernateUtil.currentSession ();
Transatcion tx = session.beginTransaction ();
for (int i = 0; i <10000; i + +)
... (
Student st = new Student ();
st.setName ( "feifei");
session.save (st);
if (i% 50 == 0) / / for every 50 data as a processing unit
... (
session.flush (); / / maintain data synchronization with the database
session.clear (); / / clear the internal cache all the data, timely release the memory occupied by
)
)
tx.commit ();
.........

The data in a certain size, this way the system memory resources can be maintained at a relatively stable range.

Note: The previously mentioned second-level cache, where the need to mention about. If you enable the second-level cache, from the mechanism in terms of Hibernate in order to maintain two cache, we have to do insert, update, delete operations, Hibernate will be filled to the second-level cache into the appropriate data. Performance, there will be a great loss, it is proposed that in a batch of second-level cache is disabled.

相关文章
  • Hibernate cache configuration / batch processing 2010-04-07

    Hibernate cache configuration / batch processing Keywords: hibernate Hibernate In addition to automatically conduct the affairs of a Session-level cache, the second-level cache will need to achieve org.hibernate.cache.CacheProvider interface, Hiberna

  • Hibernate cache configuration 2009-05-08

    Hibernate automatically Session level apart from the affairs of a cache, the secondary cache are required org.hibernate.cache.CacheProvider interface implementation, Hibernate has a number of cache implementation, developers can configure the use of

  • Spring hibernate cache configuration 2010-03-29

    Recent projects need to deal with hibernate cache: The network point of view of some articles summarized below: Great truths, principles, do not say, and write about the direct use of the. The first method: Configure Spring <property name="hiberna

  • Hibernate batch processing solution to the problem under 2011-07-19

    Many people on Java in the batch processing of data whether it is the right place skeptical of the idea, and by extension, so that the ORM will not be particularly suitable for bulk data processing. In fact, I think if we applied properly, can elimin

  • In the Hibernate batch processing in the 2010-03-29

    1: Bulk Insert In the project development process, as the project requirements, we often need large quantities of data into the database. Order of magnitude there is 10000, one hundred thousand grade, 1 million, and even 10 million level. So the numb

  • Hibernate batch processing application series of six articles 2010-08-29

    Technical documentation] [IT168 Abstract: For a long time, Hibernate batch operation as part of the performance by using groups of questions, they may still feel JDBC-processing capacity in this area should be much stronger. We all know that Hibernat

  • Hibernate batch processing (Batch processing) 2010-06-12

    Use Hibernate will insert 100,000 records into the database may be a very natural way so Session session = sessionFactory.openSession (); Transaction tx = session.beginTransaction (); for (int i = 0; i <100000; i + +) ( Customer customer = new Custom

  • Spring application of Hibernate second-level cache configuration 2010-02-14

    The first step: In the Spring configuration file by adding the following, with corresponding parameter changes, the red part is the statement using the Hibernate second-level cache, the first one to declare the use of Hibernate second-level cache, th

  • Hibernate --- caching mechanism 3 (2 cache configuration) 2010-03-27

    2.4, secondary cache configuration using the EhCache 2.4.1, configuration ready 1) the ehcache-1.2.3.jar to the classpath of the current application. 2) In the hibernate.cfg.xml file in the cache plug-in to provide added EhCache class. <!-- Configure

  • Spring + hibernate ehcache configuration to use as a second-level cache 2010-12-02

    Web application of large amounts of data flow is a common cause of performance problems, while the cache is widely used to optimize database application. cache is designed to load from the database by saving the data to reduce the use and flow of dat

  • Hibernate cache and secondary cache integrated 2010-11-28

    Transfer from http://myoraclex.blog.51cto.com/2288027/413177 1, Hibernate Caching Overview Cache is physical data sources and between applications, and is the database data stored in memory temporary copy of the container, its role is to reduce appli

  • Detailed hibernate cache 2009-04-09

    hibernate provides the level of the session cache, each session, the same id twice a load, does not send two sql to the database, but turn off session, when a cache on a failure. SessionFactory level 2 cache is global cache, it can be used under diff

  • java batch processing 2009-04-10

    Received from the Excel data: package com.mobilecn.kys.export; import java.io.File; import java.io.IOException; import java.util.ArrayList; import java.util.List; import jxl.Cell; import jxl.Sheet; import jxl.Workbook; import jxl.read.biff.BiffExcept

  • hibernate cache 2009-08-17

    hibernate The session provides a cache for each session, twice on the same id to load, does not send 2 sql to the database, but the session closed, a cache becomes ineffective. SessionFactory level 2 cache is a global cache, it can be used under diff

  • Hibernate cache Raiders II 2009-08-22

    Many people are not very familiar with the secondary cache, or there is a wrong understanding, I have always wanted to write an article tell us about the hibernate second cache, and finally help it. Mainly from my experience hibernate2.1 version of t

  • With regard to the use of hibernate cache 2009-09-04

    To reprint http://blog.csdn.net/woshichenxu/archive/2006/01/22/586361.aspx 1. Hibernate on the issue of cache: 1.1.1. The basic principle of the cache Hibernate cache is divided into two, the first-class stored in the session referred to as a cache,

  • Hibernate cache problem 2010-03-29

    1. With regard to hibernate caching problems: 1.1.1. The basic theory of caching Hibernate cache is divided into two, the first-level store in the session, called a cache, the default and can not be unloaded with. The second level is controlled by th

  • Hibernate cache discussion 2010-03-29

    There are three kinds of general system will bypass the hibernate implementation of database operations: 1, multiple applications simultaneous access to a database Such a situation would inevitably use the hibernate 2 cache of the cause data inconsis

  • Transfer] With regard to the use of hibernate cache 2010-03-24

    [Change] With regard to the use of text from hibernate cache http://blog.csdn.net/woshichenxu/archive/2006/01/22/586361.aspx 1. With regard to hibernate caching problems: 1.1.1. The basic theory of caching Hibernate cache is divided into two, the fir

  • hibernate cache related 2010-09-15

    hibernate cache related hibernate's session provided a cache of each session, twice on the same id to load, does not send 2 sql to the database, but the session closed, a cache becomes ineffective. SessionFactory level 2 cache is a global cache, it c