Working case – 第 22 页

用_minimum_giga_scn解决无法启动的数据库

2009-01-22 4条评论

今天遇到一个数据库无法启动，看到alertlog中主要是ora 600和[2662]的报错：

SQL> startup
ORACLE instance started.

Total System Global Area 1076850392 bytes
Fixed Size                   736984 bytes
Variable Size             536870912 bytes
Database Buffers          536870912 bytes
Redo Buffers                2371584 bytes
Database mounted.
ORA-01092: ORACLE instance terminated. Disconnection forced


SQL>

1

2

3

4

5

6

7

8

9

10

11

12

13

SQL> startup

ORACLE instance started.

Total System Global Area 1076850392 bytes

Fixed Size 736984 bytes

Variable Size 536870912 bytes

Database Buffers 536870912 bytes

Redo Buffers 2371584 bytes

Database mounted.

ORA-01092: ORACLE instance terminated. Disconnection forced

SQL>

其中alertlog中报错：

Completed: ALTER DATABASE   MOUNT
Thu Jan 22 13:05:08 2009
ALTER DATABASE OPEN
Thu Jan 22 13:05:09 2009
Beginning crash recovery of 1 threads
Thu Jan 22 13:05:09 2009
Started first pass scan
Thu Jan 22 13:05:09 2009
Completed first pass scan
 0 redo blocks read, 0 data blocks need recovery
Thu Jan 22 13:05:09 2009
Started recovery at
 Thread 1: logseq 2, block 3, scn 0.43536037
Recovery of Online Redo Log: Thread 1 Group 1 Seq 2 Reading mem 0
  Mem# 0 errs 0: /oracle/oradata/ora9i/redo01.log
Thu Jan 22 13:05:09 2009
Ended recovery at
 Thread 1: logseq 2, block 3, scn 0.43556038
 0 data blocks read, 0 data blocks written, 0 redo blocks read
Crash recovery completed successfully
Thu Jan 22 13:05:10 2009
Thread 1 advanced to log sequence 3
Thread 1 opened at log sequence 3
  Current log# 2 seq# 3 mem# 0: /oracle/oradata/ora9i/redo02.log
Successful open of redo thread 1.
Thu Jan 22 13:05:10 2009
SMON: enabling cache recovery
Thu Jan 22 13:05:10 2009
Errors in file /oracle/admin/ora9i/udump/ora9i_ora_12968.trc:
ORA-00600: internal error code, arguments: [2662], [0], [43556042], [261], [2396789971], [4194729], [], []

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

Completed: ALTER DATABASE MOUNT

Thu Jan 22 13:05:08 2009

ALTER DATABASE OPEN

Thu Jan 22 13:05:09 2009

Beginning crash recovery of 1 threads

Thu Jan 22 13:05:09 2009

Started first pass scan

Thu Jan 22 13:05:09 2009

Completed first pass scan

0 redo blocks read, 0 data blocks need recovery

Thu Jan 22 13:05:09 2009

Started recovery at

Thread 1: logseq 2, block 3, scn 0.43536037

Recovery of Online Redo Log: Thread 1 Group 1 Seq 2 Reading mem 0

Mem# 0 errs 0: /oracle/oradata/ora9i/redo01.log

Thu Jan 22 13:05:09 2009

Ended recovery at

Thread 1: logseq 2, block 3, scn 0.43556038

0 data blocks read, 0 data blocks written, 0 redo blocks read

Crash recovery completed successfully

Thu Jan 22 13:05:10 2009

Thread 1 advanced to log sequence 3

Thread 1 opened at log sequence 3

Current log# 2 seq# 3 mem# 0: /oracle/oradata/ora9i/redo02.log

Successful open of redo thread 1.

Thu Jan 22 13:05:10 2009

SMON: enabling cache recovery

Thu Jan 22 13:05:10 2009

Errors in file /oracle/admin/ora9i/udump/ora9i_ora_12968.trc:

ORA-00600: internal error code, arguments: [2662], [0], [43556042], [261], [2396789971], [4194729], [], []

遇到ora-600和2662的问题，我们一般 […]

..experience, Working case

flashback_on被自动改回no

2009-01-21 没有评论

oracle对flashback log采用了自动管理的方式，并且不允许手工去删除flashback log。当flashback log写满时，会自动的覆盖掉最早的一个flashback日志。并且值得注意的是，如果手工删除了flashback log，数据库将自动把flashback设置成no！ […]

..experience, Working case

HP的查看打开文件数工具:crashinfo

2009-01-17 5条评论

一天，在一个业务高峰期，某省的一个数据库突然宕机了，根据alertlog中发现的报错：

Mon Jan 12 09:49:42 2009
ARC0: Evaluating archive   log 5 thread 1 sequence 142032
ARC0: Beginning to archive log 5 thread 1 sequence 142032
Creating archive destination LOG_ARCHIVE_DEST_1: '/oraarchlog/arch_1_142032.arc'
Mon Jan 12 09:49:56 2009
ARC0: Completed archiving  log 5 thread 1 sequence 142032
Mon Jan 12 09:57:57 2009
Errors in file /oracle/app/oracle/admin/orasid/bdump/orasid_lgwr_8267.trc:
ORA-00313: open failed for members of log group 1 of thread 1
ORA-00312: online log 1 thread 1: '/dev/vg_ora3/rredo_256m_21'
ORA-27041: unable to open file
HP-UX Error: 23: File table overflow
Additional information: 2
ORA-00312: online log 1 thread 1: '/dev/vg_ora2/rredo_256m_11'
ORA-27041: unable to open file
HP-UX Error: 23: File table overflow
Additional information: 2
ORA-00312: online log 1 thread 1: '/dev/vg_ora1/rredo_256m_01'
ORA-27041: unable to open file
HP-UX Error: 23: File table overflow
Additional information: 2
Mon Jan 12 09:57:57 2009
Errors in file /oracle/app/oracle/admin/orasid/bdump/orasid_lgwr_8267.trc:
ORA-00313: open failed for members of log group 1 of thread 1
ORA-00312: online log 1 thread 1: '/dev/vg_ora3/rredo_256m_21'
ORA-27041: unable to open file
HP-UX Error: 23: File table overflow
Additional information: 2
ORA-00312: online log 1 thread 1: '/dev/vg_ora2/rredo_256m_11'
ORA-27041: unable to open file
HP-UX Error: 23: File table overflow
Additional information: 2

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

Mon Jan 12 09:49:42 2009

ARC0: Evaluating archive log 5 thread 1 sequence 142032

ARC0: Beginning to archive log 5 thread 1 sequence 142032

Creating archive destination LOG_ARCHIVE_DEST_1: '/oraarchlog/arch_1_142032.arc'

Mon Jan 12 09:49:56 2009

ARC0: Completed archiving log 5 thread 1 sequence 142032

Mon Jan 12 09:57:57 2009

Errors in file /oracle/app/oracle/admin/orasid/bdump/orasid_lgwr_8267.trc:

ORA-00313: open failed for members of log group 1 of thread 1

ORA-00312: online log 1 thread 1: '/dev/vg_ora3/rredo_256m_21'

ORA-27041: unable to open file

HP-UX Error: 23: File table overflow

Additional information: 2

ORA-00312: online log 1 thread 1: '/dev/vg_ora2/rredo_256m_11'

ORA-27041: unable to open file

HP-UX Error: 23: File table overflow

Additional information: 2

ORA-00312: online log 1 thread 1: '/dev/vg_ora1/rredo_256m_01'

ORA-27041: unable to open file

HP-UX Error: 23: File table overflow

Additional information: 2

Mon Jan 12 09:57:57 2009

Errors in file /oracle/app/oracle/admin/orasid/bdump/orasid_lgwr_8267.trc:

ORA-00313: open failed for members of log group 1 of thread 1

ORA-00312: online log 1 thread 1: '/dev/vg_ora3/rredo_256m_21'

ORA-27041: unable to open file

HP-UX Error: 23: File table overflow

Additional information: 2

ORA-00312: online log 1 thread 1: '/dev/vg_ora2/rredo_256m_11'

ORA-27041: unable to open file

HP-UX Error: 23: File table overflow

Additional information: 2

看来是打开文件数太多，导致了此次数据库宕机。通过监控打开文件数发现，其实在前一天的时候，打开文件数已经到98%，在上午8点多的时候，打开文件数已 […]

..experience, Working case

裸设备的划分

2009-01-08 没有评论

在三少的blog上看到了这篇《spfile啥都没了》。在这边和大家分享一下裸设备的划分的经验。 1.vg的属性建议用root。因为vg的宿主如果是oracle，那么在create tablespace的时候，如果没有找对正确的lv，比如我的lv名字是\dev\vg_ora1\aaa，那么在creat […]

..experience, Working case

修改sys密码导致legato无法正常备份

2009-01-06 没有评论

今天接到说某省的legato备份无法执行，查看legato的monitor窗口，发现没有明显报错，直接就出备份fail了信息了。由于monitor中说“……Hostname(s) Unresolved,1 Failed,1 Succeeded(xj_db Failed)”，一开始是怀疑hostna […]

..experience, Working case

基于catalog库进行部分表空间的还原

2009-01-04 7条评论

应某省的局方要求，在双机热备的备机上，将主机数据库的system、undo、temp和某个应用的表空间DATA_STATIC进行还原。环境如下：（1）HA主机：xj_db01 （2）HA备机：xj_db02 （3）备份软件：LEGATO 且legato的备份脚本为： [crayon-67d7c4 […]

..experience, Working case

公司招聘DBA

2008-12-26 8条评论

公司在招dba，哪位有兴趣的，可以发简历给我hejianmin[at]aspire-tech.com或者jimmyhe1981[at]gmail.com。公司名称：卓望，移动子公司，主要股东为中国移动、HP、沃达丰、美林证券地点：深圳南山区科技园人数：1人截至时间：2009年1月底前岗位职 […]

..experience, Working case

statspack中易被忽悠的地方

2008-12-15 4条评论

最近在做一个省的调优时发现该省的数据库总是不定期的出现业务堵塞，而且晚上出现的情况更加严重些。为了分析该情况，部署了一个statspack，每隔10分钟收集一次数据。发现在发生堵塞的时候，statspack中top 5 event的第一位是enqueue： [crayon-67d7c42b0a02 […]

..experience, Working case

recover时报错ora-289，有意思

2008-12-08 4条评论

oracle的alter database recover语句实在很弱智，明明目录中有arch文件，却不去找这个文件。看下面的2个例子：

-- 例子1：
sys@ORALOCAL(10.1.26.26)> alter database datafile 'e:\test02.dbf' offline;

Database altered.

Elapsed: 00:00:02.73
sys@ORALOCAL(10.1.26.26)>
sys@ORALOCAL(10.1.26.26)> alter database recover datafile 'e:\test02.dbf';

Database altered.

Elapsed: 00:01:00.25
sys@ORALOCAL(10.1.26.26)> alter database datafile 'e:\test02.dbf' online;

Database altered.

Elapsed: 00:00:00.43
sys@ORALOCAL(10.1.26.26)>

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

-- 例子1：

sys@ORALOCAL(10.1.26.26)> alter database datafile 'e:\test02.dbf' offline;

Database altered.

Elapsed: 00:00:02.73

sys@ORALOCAL(10.1.26.26)>

sys@ORALOCAL(10.1.26.26)> alter database recover datafile 'e:\test02.dbf';

Database altered.

Elapsed: 00:01:00.25

sys@ORALOCAL(10.1.26.26)> alter database datafile 'e:\test02.dbf' online;

Database altered.

Elapsed: 00:00:00.43

sys@ORALOCAL(10.1.26.26)>

-- 例子2：
sys@ORALOCAL(10.1.26.26)> alter database datafile 'e:\test02.dbf' offline;

Database altered.

Elapsed: 00:00:00.50
sys@ORALOCAL(10.1.26.26)> alter system switch logfile;

System altered.

Elapsed: 00:00:01.03
sys@ORALOCAL(10.1.26.26)> /

System altered.

Elapsed: 00:00:05.14
sys@ORALOCAL(10.1.26.26)> /

System altered.

Elapsed: 00:00:00.15
sys@ORALOCAL(10.1.26.26)> /

System altered.

Elapsed: 00:00:05.28
sys@ORALOCAL(10.1.26.26)> /

System altered.

Elapsed: 00:00:00.15
sys@ORALOCAL(10.1.26.26)> /

System altered.

Elapsed: 00:00:00.21
sys@ORALOCAL(10.1.26.26)> /

System altered.

Elapsed: 00:00:00.32
sys@ORALOCAL(10.1.26.26)> /

System altered.

Elapsed: 00:00:00.35
sys@ORALOCAL(10.1.26.26)> /

System altered.

Elapsed: 00:00:05.17
sys@ORALOCAL(10.1.26.26)> alter database recover datafile 'e:\test02.dbf';
alter database recover datafile 'e:\test02.dbf'
*
ERROR at line 1:
ORA-00279: change 801109 generated at 12/08/2008 13:21:12 needed for thread 1
ORA-00289: suggestion :
D:\ORACLE\PRODUCT\10.2.0\FLASH_RECOVERY_AREA\ORALOCAL\ARCHIVELOG\2008_12_08\O1_MF_1_55_%U_.ARC
ORA-00280: change 801109 for thread 1 is in sequence #55


Elapsed: 00:00:00.59
sys@ORALOCAL(10.1.26.26)>
sys@ORALOCAL(10.1.26.26)>
sys@ORALOCAL(10.1.26.26)> alter database recover cancel;

Database altered.

Elapsed: 00:00:01.18
sys@ORALOCAL(10.1.26.26)>
sys@ORALOCAL(10.1.26.26)>
sys@ORALOCAL(10.1.26.26)> alter database recover automatic datafile 'e:\test02.dbf';

Database altered.

Elapsed: 00:00:02.29
sys@ORALOCAL(10.1.26.26)> alter database datafile 'e:\test02.dbf' online;

Database altered.

Elapsed: 00:00:00.39
sys@ORALOCAL(10.1.26.26)>

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

-- 例子2：

sys@ORALOCAL(10.1.26.26)> alter database datafile 'e:\test02.dbf' offline;

Database altered.

Elapsed: 00:00:00.50

sys@ORALOCAL(10.1.26.26)> alter system switch logfile;

System altered.

Elapsed: 00:00:01.03

sys@ORALOCAL(10.1.26.26)> /

System altered.

Elapsed: 00:00:05.14

sys@ORALOCAL(10.1.26.26)> /

System altered.

Elapsed: 00:00:00.15

sys@ORALOCAL(10.1.26.26)> /

System altered.

Elapsed: 00:00:05.28

sys@ORALOCAL(10.1.26.26)> /

System altered.

Elapsed: 00:00:00.15

sys@ORALOCAL(10.1.26.26)> /

System altered.

Elapsed: 00:00:00.21

sys@ORALOCAL(10.1.26.26)> /

System altered.

Elapsed: 00:00:00.32

sys@ORALOCAL(10.1.26.26)> /

System altered.

Elapsed: 00:00:00.35

sys@ORALOCAL(10.1.26.26)> /

System altered.

Elapsed: 00:00:05.17

sys@ORALOCAL(10.1.26.26)> alter database recover datafile 'e:\test02.dbf';

alter database recover datafile 'e:\test02.dbf'

*

ERROR at line 1:

ORA-00279: change 801109 generated at 12/08/2008 13:21:12 needed for thread 1

ORA-00289: suggestion :

D:\ORACLE\PRODUCT\10.2.0\FLASH_RECOVERY_AREA\ORALOCAL\ARCHIVELOG\2008_12_08\O1_MF_1_55_%U_.ARC

ORA-00280: change 801109 for thread 1 is in sequence #55

Elapsed: 00:00:00.59

sys@ORALOCAL(10.1.26.26)>

sys@ORALOCAL(10.1.26.26)> alter database recover cancel;

Database altered.

Elapsed: 00:00:01.18

sys@ORALOCAL(10.1.26.26)>

sys@ORALOCAL(10.1.26.26)> alter database recover automatic datafile 'e:\test02.dbf';

Database altered.

Elapsed: 00:00:02.29

sys@ORALOCAL(10.1.26.26)> alter database datafile 'e:\test02.dbf' online;

Database altered.

Elapsed: 00:00:00.39

sys@ORALOCAL(10.1.26.26)>

可见，不加automatic […]

..experience, Working case

drop user报错ora-942

2008-11-29 没有评论

今天在一个新建的测试库上drop user的时候，报错ora-942：

sys@ORALOCAL(10.1.26.26)> drop user HEJIANM cascade;
drop user HEJIANM cascade
*
ERROR at line 1:
ORA-00604: error occurred at recursive SQL level 1
ORA-00942: table or view does not exist
ORA-06512: at line 11


Elapsed: 00:00:10.50

1

2

3

4

5

6

7

8

9

10

sys@ORALOCAL(10.1.26.26)> drop user HEJIANM cascade;

drop user HEJIANM cascade

*

ERROR at line 1:

ORA-00604: error occurred at recursive SQL level 1

ORA-00942: table or view does not exist

ORA-06512: at line 11

Elapsed: 00:00:10.50

对于942的报错，我们常用的分析方法是设置一个942的errorstack或者10046的trace，在这里，我用了10046的方式来trace这个问题： [cr […]

..db tuning, ..experience, Working case

一次cpu的user比例过高的调优

2008-11-21 4条评论

近期遇一个省的数据库说总是繁忙，idle很小，在凌晨稍微跑几个比较大的统计脚本，idle就出现0的情况，导致业务发生堵塞。经过连续几天的观察，发现cpu曲线大致如下： 2008年11月18日：和白鳝交流了一下，老白说cpu的user%比例过高可能是由于buffer get比较高的sql引起。可以 […]

..oracle service, Working case

oracle像什么？

2008-11-12 2条评论

oracle不是一个数据库公司，oracle是一个信息化公司。 oracle像一把伞，Database是伞的伞柄，Midware是伞柄上的伞骨，各类的application是伞骨上的伞面。 .