Tue Aug 17 17:34:21 2010WARNING: Oracle executable binary mismatch detected.Binary of new process does not match binary which started instanceissue alter system set "_disable_image_check" = true to disable these messagesTue Aug 17 17:34:21 2010Errors in file /u04/app/oracle/diag/rdbms/orahcmu/ORAHCMU/trace/ORAHCMU_p023_22262.trc (incident=12505):ORA-00600: internal error code, arguments: [15801], [1], [], [], [], [], [], [], [], [], [], []Incident details in: /u04/app/oracle/diag/rdbms/orahcmu/ORAHCMU/incident/incdir_12505/ORAHCMU_p023_22262_i12505.trcTue Aug 17 17:34:21 2010Errors in file /u04/app/oracle/diag/rdbms/orahcmu/ORAHCMU/trace/ORAHCMU_p021_22258.trc (incident=12489):ORA-00600: internal error code, arguments: [15801], [1], [], [], [], [], [], [], [], [], [], []Incident details in: /u04/app/oracle/diag/rdbms/orahcmu/ORAHCMU/incident/incdir_12489/ORAHCMU_p021_22258_i12489.trcErrors in file /u04/app/oracle/diag/rdbms/orahcmu/ORAHCMU/trace/ORAHCMU_p015_9328.trc (incident=19909):ORA-00600: internal error code, arguments: [15801], [1], [], [], [], [], [], [], [], [], [], []Errors in file /u04/app/oracle/diag/rdbms/orahcmu/ORAHCMU/trace/ORAHCMU_p043_9388.trc (incident=20133):ORA-00600: internal error code, arguments: [15801], [1], [], [], [], [], [], [], [], [], [], []Mon Aug 23 14:43:42 2010Errors in file /u04/app/oracle/diag/rdbms/orahcmu/ORAHCMU/trace/ORAHCMU_p087_9668.trc (incident=20485):ORA-00600: internal error code, arguments: [15801], [1], [], [], [], [], [], [], [], [], [], []Mon Aug 23 14:43:42 2010Errors in file /u04/app/oracle/diag/rdbms/orahcmu/ORAHCMU/trace/ORAHCMU_p012_9322.trc (incident=19885):ORA-00600: internal error code, arguments: [15801], [1], [], [], [], [], [], [], [], [], [], []Incident details in: /u04/app/oracle/diag/rdbms/orahcmu/ORAHCMU/incident/incdir_19789/ORAHCMU_ora_8602_i19789.trcMon Aug 23 14:43:43 2010WARNING: Oracle executable binary mismatch detected.Binary of new process does not match binary which started instanceissue alter system set "_disable_image_check" = true to disable these messagesDump continued from file: /u04/app/oracle/diag/rdbms/orahcmu/ORAHCMU/trace/ORAHCMU_ora_8602.trcORA-00600: internal error code, arguments: [15801], [1], [], [], [], [], [], [], [], [], [], []*** 2010-08-23 14:43:42.974----- Current SQL Statement for this session (sql_id=00abhfx460qm9) -----CREATE UNIQUE iNDEX PS_HM_BEN_GP_STG ON PS_HM_BEN_GP_STG (CAL_ID, GP_PAYGROUP,EMPLID, EMPL_RCD, HM_INCURRED_BY, HM_SUM_ASSURED) TABLESPACE PSINDEX STORAGE(INITIAL 40000 NEXT 100000 MAXEXTENTS UNLIMITED PCTINCREASE 0) PCTFREE 10 PARALLEL NOLOGGING----- Call Stack Trace -----ksedst1 ksedst dbkedDefDump dbgexPhaseII dbgexProcessError dbgePostErrorKGE kgeade kgeremkxfpProcessError kxfpqidqr kxfpqdqr kxfxgs kxfxcp qerpxSendParse kxfpValidateSlaveGroup kxfpgsgkxfrAllocSlaves kxfrialo kxfralo qerpx_rowsrc_start qerpxStart kdicrws kdicdrv opiexe opiosq0kpooprx kpoal8 opiodr ttcpip opitsk opiino opiodr opidrv sou2o mainSO: 0x3bf0bbf20, type: 4, owner: 0x3bf5452d0, flag: INIT/-/-/0x00 if: 0x3 c: 0x3proc=0x3bf5452d0, name=session, file=ksu.h LINE:10719 ID:, pg=0(session) sid: 217 ser: 767 trans: 0x3bc0660f8, creator: 0x3bf5452d0flags: (0x8000041) USR/- flags_idl: (0x1) BSY/-/-/-/-/-flags2: (0x44008) DDLT1/-DID: , short-term DID:txn branch: 0x0oct: 9, prv: 0, sql: 0x3b5d14510, psql: 0x3b6d59820, user: 31/SYSADMksuxds FALSE at location: 0service name: ORAHCMUclient details:O/S info: user: Administrator, term: UJWALTPVM, ospid: 304:2892machine: WORKGROUP\UJWALTPVM program: pside.execlient info: ujwal,Administrator,UJWALTPVM,,pside.exe,application name: pside.exe, hash value=2824484291Current Wait Stack:Not in wait; last wait ended 2.475286 sec agoWait State:auto_close=0 flags=0x21 boundary=0x0/-1Session Wait History:0: waited for 'lient'=c8, =1, =0wait_id=10483 seq_num=10484 snap_id=1wait times: snap=0.168502 sec, exc=0.168502 sec, total=0.168502 secwait times: max=2.000000 secwait counts: calls=1 os=1occurred after 0.000903 sec of elapsed time1: waited for ' waiting for ruleset'=10010063, =1, =0wait_id=10482 seq_num=10483 snap_id=1wait times: snap=0.008580 sec, exc=0.008580 sec, total=0.008580 secwait times: max=2.000000 secwait counts: calls=1 os=1occurred after 0.000731 sec of elapsed time2: waited for ' waiting for ruleset'=1001004f, =4, =0wait_id=10481 seq_num=10482 snap_id=1wait times: snap=0.000132 sec, exc=0.000132 sec, total=0.000132 secwait times: max=2.000000 secwait counts: calls=1 os=1occurred after 0.000074 sec of elapsed time3: waited for ' waiting for ruleset'=1001004f, =3, =0wait_id=10480 seq_num=10481 snap_id=1wait times: snap=0.000002 sec, exc=0.000002 sec, total=0.000002 secwait times: max=2.000000 secwait counts: calls=1 os=1occurred after 0.000065 sec of elapsed time----- Session Cursor Dump -----Current cursor: 1, pgadep=0Open cursors(pls, sys, hwm, max): 3(0, 2, 64, 300)NULL=1 SYNTAX=0 PARSE=0 BOUND=1 FETCH=0 ROW=1Cached frame pages(total, free):4k(14, 14), 8k(1, 1), 16k(1, 1), 32k(0, 0)----- Current Cursor ---------- Plan Table -----============Plan Table============----------------------------------------------------+-----------------------------------+-------------------------+| Id | Operation | Name | Rows | Bytes | Cost | Time | TQ |IN-OUT|PQ Distrib |----------------------------------------------------+-----------------------------------+-------------------------+| 0 | CREATE INDEX STATEMENT | | | | 2 | | | | || 1 | PX COORDINATOR | | | | | | | | || 2 | PX SEND QC (ORDER) | :TQ10001 | 82 | 4510 | | |:Q1001| P->S |QC (ORDER) || 3 | INDEX BUILD UNIQUE | PS_HM_BEN_GP_STG| | | | |:Q1001| PCWP | || 4 | SORT CREATE INDEX | | 82 | 4510 | | |:Q1001| PCWP | || 5 | PX RECEIVE | | 82 | 4510 | 2 | 00:00:01 |:Q1001| PCWP | || 6 | PX SEND RANGE | :TQ10000 | 82 | 4510 | 2 | 00:00:01 |:Q1000| P->P |RANGE || 7 | PX BLOCK ITERATOR | | 82 | 4510 | 2 | 00:00:01 |:Q1000| PCWC | || 8 | TABLE ACCESS FULL | PS_HM_BEN_GP_STG| 82 | 4510 | 2 | 00:00:01 |:Q1000| PCWP | |----------------------------------------------------+-----------------------------------+-------------------------+----------------------------------------Cursor#1(0xffffffff7ce31928) state=BOUND curiob=0xffffffff7ce57d28curflg=4c fl2=0 par=0x0 ses=0x3bf0bbf20----- Dump Cursor sql_id=00abhfx460qm9 xsc=0xffffffff7ce57d28 cur=0xffffffff7ce31928 -----Dump Parent Cursor sql_id=00abhfx460qm9 phd=0x3b5d14510 plk=0x3b0bb3318sqltxt(0x3b5d14510)=CREATE UNIQUE iNDEX PS_HM_BEN_GP_STG ON PS_HM_BEN_GP_STG(CAL_ID, GP_PAYGROUP, EMPLID, EMPL_RCD, HM_INCURRED_BY, HM_SUM_ASSURED)TABLESPACE PSINDEX STORAGE (INITIAL 40000 NEXT 100000 MAXEXTENTS UNLIMITED PCTINCREASE 0)PCTFREE 10 PARALLEL NOLOGGINGhash=616eaa631fc21f4c0029707748605a69parent=0x3ae539590 maxchild=01 plk=0x3b0bb3318 ppn=ncursor instantiation=0xffffffff7ce57d28 used=1282545779 exec_id=16777216 exec=1child#0(0x3b5d05e10) pcs=0x3b678c128clk=0x3b7e200d0 ci=0x3b5b204c8 pn=0x39955d2b8 ctx=0x3b86ee988kgsccflg=0 llk[0xffffffff7ce57d30,0xffffffff7ce57d30] idx=0xscflg=c0102276 fl2=c000400 fl3=2202008 fl4=100Frames pfr 0xffffffff7ce67098 siz=85976 efr 0xffffffff7ce66fb8 siz=85960Cursor frame dumpenxt: 7.0x00000168 enxt: 6.0x00008000 enxt: 5.0x00008000 enxt: 4.0x00003978enxt: 3.0x00000490 enxt: 2.0x000000b8 enxt: 1.0x00000fa0pnxt: 1.0x00000010kxscphp=0xffffffff7dd80a18 siz=984 inu=312 nps=312kxscwhp=0xffffffff7ddd2cc8 siz=8136 inu=6264 nps=3968kxscefhp=0xffffffff7ce51468 siz=88456 inu=86128 nps=86128FileName----------------ORAHCMU_ora_8602.trcFileComment----------------------Oracle Support - August 27, 2010 6:13:39 PM GMT+08:00 [ODM Data Collection]Name--------=== ODM Data Collection ====== ODM Data Collection ===Trace file /u04/app/oracle/diag/rdbms/orahcmu/ORAHCMU/trace/ORAHCMU_p012_9322.trc*** 2010-08-23 14:43:00.472WARNING: Oracle executable binary mismatch detected.Binary of new process does not match binary which started instanceissue alter system set "_disable_image_check" = true to disable these messagesstartup image informationiid info sz=245752512 inode=65458 ts=0x4c6df668current process image informationiid info sz=245750720 inode=65427 ts=0x4c7204b0set _disable_image_check = TRUE to disable this checkqksceLinearToCe error*** 2010-08-23 14:43:42.974*** SESSION ID:(220.111) 2010-08-23 14:43:42.974*** CLIENT ID:(ujwal) 2010-08-23 14:43:42.974*** SERVICE NAME:(ORAHCMU) 2010-08-23 14:43:42.974DDE: Problem Key 'ORA 600 [15801]' was flood controlled (0x6) (incident: 19885)ORA-00600: internal error code, arguments: [15801], [1], [], [], [], [], [], [], [], [], [], []kxfxdssKXFXSLAVESTATE dump [0, 0](pgakid: 0 oercnt: 0 oerrcd: -2224892588)kxfxdssno current cursor context.kxfxdssno cursors.

The ORA-600 15801 is reporting a communication problem between QC and slaves related with messages sent/received.Alert log reports several of the following error on the ASM instance:ORA-600: internal error code, arguments: [15801], [1], [], [], [], [], [],[]last wait was for 'eq: Msg Fragment'DIAGNOSTIC ANALYSIS:--------------------There were also several of the following message in the alert log:WARNING: Oracle executable binary mismatch detected. Binary of new process does not match binary which started instanceissue alter system set "_disable_image_check" = true to disable thesemessagesSo, I asked the customer to set the "_disable_image_check" = trueThis had no impact on the ora-600 errors as expected.ORA-600 [15801] is signalled when a message overflow occurs between  PQprocesses.WORKAROUND:-----------noneRELATED BUGS:-------------noneREPRODUCIBILITY:----------------intermittent but frequently - occurs at all different times of the day.STACK TRACE:------------*** ID:(29.2904) 2006-07-05 15:50:57.972qksceLinearToCe error*** 15:50:58.233ksedmp: internal or fatal errorORA-600: internal error code, arguments: [15801], [1], [], [], [], [], [],[]----- Call Stack Trace -----kxfxGeter qks3tttdefReceive kxfxsui kxfxsp kxfxmai kxfprdp    SO: 0x67977018, type: 4, owner: 0x6793f208, flag: INIT/-/-/0x00    (session) sid: 29 trans: (nil), creator: 0x6793f208, flag: (c0000041)USR/- BSY/-/-/-/-/-              DID: 0000-0012-0000FADB, short-term DID: 0000-0000-00000000              txn branch: (nil)              oct: 3, prv: 0, sql: (nil), psql: (nil), user: 0/SYS    O/S info: user: oracle, term: , ospid: 4558, machine:    last wait for 'eq: Msg Fragment' blocking sess=0x(nil) seq=2wait_time=4441 seconds since wait started=3                ct path write=1002ffff, ct path write temp=2, Network=0    Dumping Session Wait History     for 'eq: Msg Fragment' count=1 wait_time=4441                ct path write=1002ffff, ct path write temp=2, Network=0     for 'eq: Msg Fragment' count=1 wait_time=31                ct path write=1002ffff, ct path write temp=1, Network=0    temporary object counter: 0

// 0x20 large allocation pool

本文转自maclean_007 51CTO博客，原文链接: