Sunday, February 10, 2013

Device Mapping and Logical System Board Numbers in SPARC Enterprise M3000/M4000/M5000/M8000/M9000 servers from Oracle and Fujitsu

SPARC Enterprise M3000/M4000/M5000/M8000/M9000 Servers Administration Guide

Mapping Device Path Names
This appendix describes how to map device path names to physical system devices in SPARC Enterprise M3000/M4000/M5000/M8000/M9000 servers from Oracle and Fujitsu. It contains these sections:

Device Mapping and Logical System Board Numbers

The physical address represents a physical characteristic that is unique to the device. Examples of physical addresses include the bus address and the slot number. The slot number indicates where the device is installed.
The logical system board (LSB) number affects both the processor numbering and the I/O device paths in the server. Physical resources are assigned to LSBs in the specified domain using the setdcl command. It is the LSB numbers that the Oracle Solaris OS uses.

CPU Mapping

Each LSB on a midrange or high-end server has a bank of 32 processor numbers assigned to it. For entry-level servers, the bank holds eight processors. The SPARC64 VI processor has two cores, each with two threads (also known as virtual processors). The SPARC64 VII+ and SPARC64 VII processor has two cores or four cores, each with two threads.
An LSB on a midrange or high-end server has up to four processors (when a Uni-XSB is assigned to the LSB); therefore, the LSB needs 32 processor IDs. An LSB on an entry-level server, which supports only a single processor, requires only eight processor IDs.
TABLE A-1 shows the relationship between LSB numbers and starting processor (proc) numbers, in hexadecimal/decimal format. The Oracle Solaris prtdiag(IM) command provides the LSB numbers and CPU chip numbers in decimal format for components that are part of the domain.

TABLE A-1 LSB Numbers and Starting Processor Numbers
LSB Number
CPU Chip 0
CPU Chip 1
CPU Chip 2
CPU Chip 3
Entry-Level Servers
00
00/00
N/A
N/A
N/A
Midrange and High-End Servers
00
00/00
08/08
10/16
18/24
01
20/32
28/40
30/48
38/56
02
40/64
48/72
50/80
58/88
03
60/96
68/104
70/112
78/120
04
80/128
88/136
90/144
98/152
05
a0/160
a8/168
b0/176
b8/184
06
c0/192
c8/200
d0/208
d8/216
07
e0/224
e8/232
f0/240
f8/248
08
100/256
108/264
110/272
118/280
09
120/288
128/296
130/304
138/312
10
140/320
148/328
150/336
158/344
11
160/352
168/360
170/368
178/376
12
180/384
188/392
190/400
198/408
13
1a0/416
1a8/424
1b0/432
1b8/440
14
1c0/448
1c8/456
1d0/464
1d8/472
15
1e0/480
1e8/488
1f0/496
1f8/504

CPU Numbering Examples

This section contains examples of CPU numbering, using the output of the showboards(8) command on the Service Processor, and the output of the prtdiag(1M) command on the domain. 

XSCF> showboards -a 
 
XSB  DID(LSB) Assignment  Pwr  Conn Conf Test    Fault    
---- -------- ----------- ---- ---- ---- ------- -------- 
00-0 00(00)   Assigned    y    y    y    Passed  Normal   
00-1 00(01)   Assigned    y    y    y    Passed  Normal   
00-2 00(04)   Assigned    y    y    y    Passed  Normal   
00-3 00(05)   Assigned    y    n    n    Passed  Normal   
01-0 01(00)   Assigned    y    y    y    Passed  Normal   
01-1 01(09)   Assigned    y    y    y    Passed  Normal   
01-2 01(06)   Assigned    y    n    n    Passed  Normal   
01-3 01(07)   Assigned    y    n    n    Passed  Normal   



domain_0# prtdiag -v
...
==================================== CPUs ====================================
 
       CPU              CPU            Run       L2$       CPU      CPU 
LSB    Chip              ID            MHz        MB       Impl.    Mask 
---    ----      --------------------  ----      ---       -----    ---- 
 00      0          0,   1,   2,   3   2150      4.0          6      129
 01      1         40,  41,  42,  43   2150      4.0          6      129
 04      2        144, 145, 146, 147   2150      4.0          6      129 
 05      3        184, 185, 186, 187   2150      4.0          6      129
 
==============================================================================



domain_1# prtdiag -v
...
==================================== CPUs ====================================
 
       CPU              CPU            Run       L2$       CPU      CPU 
LSB    Chip              ID            MHz        MB       Impl.    Mask 
---    ----      --------------------  ----      ---       -----    ---- 
 00      0          0,   1,   2,   3   2150      4.0          6      129
 09      1        296, 297, 298, 299   2150      4.0          6      129
 06      2        208, 209, 210, 211   2150      4.0          6      129
 07      3        248, 249, 250, 251   2150      4.0          6      129
 
==============================================================================

I/O Device Mapping

I/O device paths are dictated by which LSB the I/O unit is assigned to.
Entry-level servers have one I/O controller. The XSB is assigned four PCIe slots.
Midrange servers have only one I/O controller on the I/O unit (IOU). For an XSB in Uni-XSB mode, all I/O is on XSB#xx-0. For an XSB in Quad-XSB mode, internal resources, the PCI-X slot, and two PCIe slots are on XSB#xx-0, and two PCIe slots are on XSB#xx-1.
High-end servers have two I/O controllers; therefore, each XSB can have two PCIe slots assigned to it.
TABLE A-2 shows the LSB numbers and the corresponding device path values that are used in I/O device mapping on the server.

TABLE A-2 LSB Numbers and Device Path Values
LSB Number
Device Path Value
00
No value
01
1
02
2
03
3
04
4
05
5
06
6
07
7
08
8
09
9
10
a
11
b
12
c
13
d
14
e
15
f

I/O Device Mapping on Entry-Level Servers

TABLE A-3 shows the device mapping on an entry-level server.

TABLE A-3 I/O Device Mapping on an Entry-level Server
PCIe Slot
Host Bus Adapter Slot Type
OpenBoot PROM Device Path
Slot 0
PCIe
/pci@0,600000/pci@0/pci@8
Slot 1
PCIe
/pci@1,700000/pci@0/pci@0
Slot 2
PCIe
/pci@1,700000/pci@0/pci@8
Slot 3
PCIe
/pci@1,700000/pci@0/pci@9

Internal Devices on Entry-Level Servers With SPARC64 VII+ Processors

The entry-level server has a single system board, at location XSB 00-0. Internal devices and device paths are shown in TABLE A-4.

TABLE A-4 Internal Devices and Device Paths on an Entry-level Server with SPARC64 VII+ Processors
XSB 00-0/IOU 0
Accessible Internal Devices
Device Physical Location
OpenBoot PROM Device Path
Network Port 0
System
/pci@0,600000/pci@0/pci@1/network@0
Network Port 1
System
/pci@0,600000/pci@0/pci@1/network@0,1
Network Port 2
System
/pci@0,600000/pci@0/pci@2/network@0
Network Port 3
System
/pci@0,600000/pci@0/pci@2/network@0,1
HD0
System
/pci@0,600000/pci@0/pci@0/scsi@0/disk@0
HD1
System
/pci@0,600000/pci@0/pci@0/scsi@0/disk@1
HD2
System
/pci@0,600000/pci@0/pci@0/scsi@0/disk@2
HD3
System
/pci@0,600000/pci@0/pci@0/scsi@0/disk@3
CD-RW/DVD-RW
System
/pci@0,600000/pci@0/pci@0/scsi@0/disk@4
SAS port
System
/pci@0,600000/pci@0/pci@0/scsi@0/xx@5,z, where xx is the disk when connecting to a disk, or a tape when connecting to a tape drive unit.

Internal Devices on Entry-Level Servers Without SPARC64 VII+ Processors

The entry-level server has a single system board, at location XSB 00-0. Internal devices and device paths are shown in TABLE A-5

TABLE A-5 Internal Devices and Device Paths on an Entry-level Server without SPARC64 VII+ Processors
XSB 00-0/IOU 0
Accessible Internal Devices
Device Physical Location
OpenBoot PROM Device Path
Network Port 0
System
/pci@0,600000/pci@0/pci@1/pci@0/network@4
Network Port 1
System
/pci@0,600000/pci@0/pci@1/pci@0/network@4,1
Network Port 2
System
/pci@0,600000/pci@0/pci@2/pci@0/network@4
Network Port 3
System
/pci@0,600000/pci@0/pci@2/pci@0/network@4,1
HD0
System
/pci@0,600000/pci@0/pci@0/scsi@0/disk@0
HD1
System
/pci@0,600000/pci@0/pci@0/scsi@0/disk@1
HD2
System
/pci@0,600000/pci@0/pci@0/scsi@0/disk@2
HD3
System
/pci@0,600000/pci@0/pci@0/scsi@0/disk@3
CD-RW/DVD-RW
System
/pci@0,600000/pci@0/pci@0/scsi@0/disk@4
SAS port
System
/pci@0,600000/pci@0/pci@0/scsi@0/xx@5,z, where xx is the disk when connecting to a disk, or a tape when connecting to a tape drive unit.

I/O Device Mapping on Midrange Servers

TABLE A-6 shows the device mapping on a midrange server. In the device path, x is LSB-dependent, and is assigned a value as shown in TABLE A-2.

TABLE A-6 I/O Device Mapping on a Midrange Server
Slot
Host Bus Adapter Slot Type
OpenBoot PROM Device Path
IOU Slot 0
PCI-X
/pci@x0,600000/pci@0/pci@8/pci@0,1
IOU Slot 1
PCIe
/pci@x0,600000/pci@0/pci@9
IOU Slot 2
PCIe
/pci@x1,700000
IOU Slot 3
PCIe
/pci@x2,600000
IOU Slot 4
PCIe
/pci@x3,700000

Internal Devices on Midrange Servers

The internal midrange server devices, which are located at the XSB location 00-0 or 01-0 (regardless of Uni-XSB or Quad-XSB mode), are shown in TABLE A-7 and TABLE A-8. In the device path, x is LSB-dependent, and is assigned a value as shown in TABLE A-2.

TABLE A-7 Internal Devices and Device Paths on the Midrange Servers, IOU#0
XSB 00-0/IOU 0 Accessible Internal Devices (M4000/M5000)
Device Physical Location
OpenBoot PROM Device Path
Network Port 0
IOU#0
/pci@x0,600000/pci@0/pci@8/pci@0/network@2
Network Port 1
IOU#0
/pci@x0,600000/pci@0/pci@8/pci@0/network@2,1
HD0
System
/pci@x0,600000/pci@0/pci@8/pci@0/scsi@1/disk@0
HD1
System
/pci@x0,600000/pci@0/pci@8/pci@0/scsi@1/disk@1
CD-RW/DVD-RW
System
/pci@x0,600000/pci@0/pci@8/pci@0/scsi@1/disk@3
Tape
System
/pci@x0,600000/pci@0/pci@8/pci@0/scsi@1/tape@2


TABLE A-8 Internal Devices and Device Paths on the M5000 (but not M4000) Server, IOU#1
XSB 01-0/IOU 1 Accessible Internal Device (M5000)
Device Physical Location
OpenBoot PROM Device Path
Network Port 0
IOU#1
/pci@x0,600000/pci@0/pci@8/pci@0/network@2
Network Port 1
IOU#1
/pci@x0,600000/pci@0/pci@8/pci@0/network@2,1
HD2
System
/pci@x0,600000/pci@0/pci@8/pci@0/scsi@1/disk@0
HD3
System
/pci@x0,600000/pci@0/pci@8/pci@0/scsi@1/disk@1

I/O Device Mapping on High-End Servers

TABLE A-9 shows the device mapping on a high-end server. In the PCIe device path, x is LSB-dependent, and is assigned a value as shown in TABLE A-2. xx is the XSB number and is in the range from 00-15.

TABLE A-9 I/O Device Mapping on a High-end Server
PCIe Slot
Uni-XSB[1]
Quad-XSB[2]
OpenBoot PROM PCIe Device Path[3]
IOU Slot 0
xx-0
xx-0
pci@x0,600000
IOU Slot 1
xx-0
xx-0
pci@x1,700000
IOU Slot 2
xx-0
xx-1
pci@x2,600000
IOU Slot 3
xx-0
xx-1
pci@x3,700000
IOU Slot 4
xx-0
xx-2
pci@x4,600000
IOU Slot 5
xx-0
xx-2
pci@x5,700000
IOU Slot 6
xx-0
xx-3
pci@x6,600000
IOU Slot 7
xx-0
xx-3
pci@x7,700000

Internal Devices on High-End Servers

The IOUA is a PCIe Host Bus Adapter that provides access to internal devices when installed at specific locations. The IOUA contains two 1Gb Ethernet ports on the card (“on-board”). When the IOUA is installed at specific locations, it also provides access to storage located on the IOU, as well as platform CD-RW/DVD-RW drive or tape drive resources at the locations shown in TABLE A-10. In the PCIe device path, x is LSB-dependent, and is assigned a value as shown in TABLE A-2. xx is the XSB number and is in the range from 00-15. nn is the number associated with the PSB to which the CD-RW/DVD-RW drive or tape drive is attached, as further explained in the table footnote.
 

TABLE A-10 Internal Devices and Device Paths on a High-end Server
PCIe Slot
Uni-XSB[4]
Quad-XSB[5]
OpenBoot PROM PCIe Device Path[6]
OpenBoot PROM IOUA HBA On-board, IOU, and Platform Accessible Devices[7]
IOU Slot 0
xx-0
xx-0
pci@x0,600000
.../pci@0,1/network@1 (IOUA HBA On-board BGE Port 0)
.../pci@0,1/network@1,1 (IOUA HBA On-board BGE Port 1)
.../pci@0/scsi@1/disk@0 (IOU HD0; SCSI Target 0)
.../pci@0/scsi@1/disk@1 (IOU HD1; SCSI Target 1)
.../pci@0/scsi@1/disk@4 (Platform CD-RW/DVD-RW at cfgdevice port
nn-0; SCSI Target 4)
.../pci@0/scsi@1/tape@5 (Platform tape at cfgdevice port nn-0;
SCSI Target 5)
IOU Slot 1
xx-0
xx-0
pci@x1,700000
 
IOU Slot 2
xx-0
xx-1
pci@x2,600000
.../pci@0,1/network@1 (IOUA HBA On-board BGE Port 0)
.../pci@0,1/network@1,1 (IOUA HBA On-board BGE Port 1)
.../pci@0/scsi@1/disk@4 (Platform CD-RW/DVD-RW at cfgdevice port
nn-2; SCSI Target 4)
.../pci@0/scsi@1/tape@5 (Platform tape at cfgdevice port nn-2;
SCSI Target 5)
IOU Slot 3
xx-0
xx-1
pci@x3,700000
.
IOU Slot 4
xx-0
xx-2
pci@x4,600000
.../pci@0,1/network@1 (IOUA HBA On-board BGE Port 0)
.../pci@0,1/network@1,1 (IOUA HBA On-board BGE Port 1)
.../pci@0/scsi@1/disk@0 (IOU HD2; SCSI Target 0)
.../pci@0/scsi@1/disk@1 (IOU HD3; SCSI Target 1)
.../pci@0/scsi@1/disk@4 (Platform CD-RW/DVD-RW at cfgdevice port
nn-4; SCSI Target 4)
.../pci@0/scsi@1/tape@5 (Platform tape at cfgdevice port nn-4;
SCSI Target 5)
IOU Slot 5
xx-0
xx-2
pci@x5,700000
 
IOU Slot 6
xx-0
xx-3
pci@x6,600000
.../pci@0,1/network@1 (IOUA HBA On-board BGE Port 0)
.../pci@0,1/network@1,1 (IOUA HBA On-board BGE Port 1)
.../pci@0/scsi@1/disk@4 (Platform CD-RW/DVD-RW at cfgdevice port
nn-6; SCSI Target 4)
.../pci@0/scsi@1/tape@5 (Platform tape at cfgdevice port nn-6;
SCSI Target 5)
IOU Slot 7
xx-0
xx-3
pci@x7,700000
 

Sample cfgadm Output

This section contains:
  • Sample output for the command cfgadm -s “select=class(pci)” on an unpopulated server. As you connect devices, the cfgadm output will change to reflect the device type and connection status on your server.
  • The device matrix for midrange and for high-end servers, when the IOU is configured as part of a domain. I/O portions of the IOU resources may be in different domains.

Entry-Level Server

The entry-level server does not support PCI hotplug. Therefore, the concepts of attachment points and classes do not apply, and executing the command

cfgadm -s "select=class(pci)"
either would produce an error or display nothing.

Midrange Servers

M4000 Server sample output: 

# cfgadm -s "select=class(pci)"
Ap_Id                          Type         Receptacle   Occupant     Condition
iou#0-pci#0                    unknown      empty        unconfigured unknown
iou#0-pci#1                    unknown      empty        unconfigured unknown
iou#0-pci#2                    unknown      empty        unconfigured unknown
iou#0-pci#3                    unknown      empty        unconfigured unknown
iou#0-pci#4                    unknown      empty        unconfigured unknown

M5000 Server sample output: 

# cfgadm -s "select=class(pci)"
Ap_Id                          Type         Receptacle   Occupant     Condition
iou#0-pci#0                    unknown      empty        unconfigured unknown
iou#0-pci#1                    unknown      empty        unconfigured unknown
iou#0-pci#2                    unknown      empty        unconfigured unknown
iou#0-pci#3                    unknown      empty        unconfigured unknown
iou#0-pci#4                    unknown      empty        unconfigured unknown
iou#1-pci#0                    unknown      empty        unconfigured unknown
iou#1-pci#1                    unknown      empty        unconfigured unknown
iou#1-pci#2                    unknown      empty        unconfigured unknown
iou#1-pci#3                    unknown      empty        unconfigured unknown
iou#1-pci#4                    unknown      empty        unconfigured unknown


TABLE A-11 cfgadm Device Matrix for Midrange Servers
PCI Slot #
PCI Slot Type
IOU#0 (M4000/M5000)
IOU#1 (M5000)
0
PCI-X
iou#0-pci#0
iou#1-pci#0
1
PCIe
iou#0-pci#1
iou#1-pci#1
2
PCIe
iou#0-pci#2
iou#1-pci#2
3
PCIe
iou#0-pci#3
iou#1-pci#3
4
PCIe
iou#0-pci#4
iou#1-pci#4

High-End Servers

M8000 Server sample output: 

# cfgadm -s "select=class(pci)"
Ap_Id                          Type         Receptacle   Occupant     Condition
iou#1-pci#0                    unknown      empty        unconfigured unknown
iou#1-pci#1                    unknown      empty        unconfigured unknown
iou#1-pci#4                    unknown      empty        unconfigured unknown
iou#1-pci#5                    unknown      empty        unconfigured unknown
iou#1-pci#6                    unknown      empty        unconfigured unknown
iou#1-pci#7                    unknown      empty        unconfigured unknown

M9000 Server sample output: 

# cfgadm -s "select=class(pci)"
Ap_Id                          Type         Receptacle   Occupant     Condition
iou#0-pci#0                    unknown      empty        unconfigured unknown
iou#0-pci#1                    unknown      empty        unconfigured unknown
iou#0-pci#2                    unknown      empty        unconfigured unknown
iou#0-pci#3                    unknown      empty        unconfigured unknown
iou#0-pci#4                    unknown      empty        unconfigured unknown
iou#0-pci#5                    unknown      empty        unconfigured unknown
iou#0-pci#6                    unknown      empty        unconfigured unknown
iou#0-pci#7                    unknown      empty        unconfigured unknown
iou#3-pci#0                    unknown      empty        unconfigured unknown
iou#3-pci#1                    unknown      empty        unconfigured unknown
iou#3-pci#2                    unknown      empty        unconfigured unknown
iou#3-pci#3                    unknown      empty        unconfigured unknown


TABLE A-12 cfgadm Device Matrix for High-End Servers
PCI Slot #
PCI Slot Type
IOU#0
IOU#1
IOU#n[8]
0
PCIe
iou#0-pci#0
iou#1-pci#0
iou#n-pci#0
1
PCIe
iou#0-pci#1
iou#1-pci#1
iou#n-pci#1
2
PCIe
iou#0-pci#2
iou#1-pci#2
iou#n-pci#2
3
PCIe
iou#0-pci#3
iou#1-pci#3
iou#n-pci#3
4
PCIe
iou#0-pci#4
iou#1-pci#4
iou#n-pci#4
5
PCIe
iou#0-pci#5
iou#1-pci#5
iou#n-pci#5
6
PCIe
iou#0-pci#6
iou#1-pci#6
iou#n-pci#6
7
PCIe
iou#0-pci#7
iou#1-pci#7
iou#n-pci#7

1 (TableFootnote) xx is the XSB number, 00-15
2 (TableFootnote) xx is the XSB number, 00-15
3 (TableFootnote) x is LSB-dependent, assigned a value as shown in TABLE A-2
4 (TableFootnote) xx is the XSB number, in the range of 00-15.
5 (TableFootnote) xx is the XSB number, in the range of 00-15.
6 (TableFootnote) x is LSB-dependent, and is assigned a value as shown in TABLE A-2.
7 (TableFootnote) nn is the number associated with the PSB to which the CD-RW/DVD-RW drive or tape drive is attached, as follows: for an M8000 server, nn is in the range of 0-3; for an M9000 server, nn is in the range of 0-7; for an M9000 server plus expansion unit, nn is in the range of 0-15.
8 (TableFootnote) n is the IOU number


SPARC Enterprise M3000/M4000/M5000/M8000/M9000 Servers Administration Guide

Posted by at

24 comments:

  1. UnknownFebruary 8, 2016 at 11:45 PM


    Very nice I gathered good information from this content.
    HTML5 Training in Chennai | HTML5 Training institute in Chennai | Fita Training.

    ReplyDelete
  2. UnknownNovember 3, 2016 at 10:35 PM

    Nice post. This is truly a great post.
    unix training in chennai

    ReplyDelete
  3. asp .net web development companyMay 16, 2017 at 9:00 AM

    Awesome blog,Thanks for sharing useful information article keep sharing knowledge with us it's very nice.
    web designing services in chennai.
    best web design company in chennai.

    ReplyDelete
  4. asp .net web development companyMay 24, 2017 at 4:18 AM

    Thanks for sharing useful information article to us keep sharing knowledge with us,
    Hi We at Colan Infotech Private Limited a
    Mobile application development company in chennai,
    is Situated in US and India, will provide you best service in
    enterprise mobile app development company .
    and Colan Infotech has a group of exceedingly dedicated, inventive and creative experts with an energy for delivering

    exciting , helpful and stylish Web and Mobile Applications, We work with customers in a wide variety of sectors.
    We design all of our websites and applications using the responsive web design approach. Our talented team can handle all

    the aspects of mobility so we are rated as best service provider in
    Mobile apps development companies in chennai.
    We solidly trust that our customers start things out and there is not a viable alternative for quality of service.
    We offer custom services to a wide range of industries by exceeding our client’s expectations. You can even interact

    directly with the team regarding your project, just as you would with your in-house team. we always desire to solicit our

    customer's fruitful experience with us, we are the top notch
    Mobile App Development Company in chennai
    and mobile app development companies in Bangalore. We can provide best
    mobile app development chennai .
    We can provide cutting edge technology services in
    Mobile application development in chennai.
    Reach us for mobile app development chennai or just call us for best
    mobile app developers in chennai .

    ReplyDelete
  5. gowsalyaAugust 30, 2018 at 5:13 AM

    Existing without the answers to the difficulties you’ve sorted out through this guide is a critical case, as well as the kind which could have badly affected my entire career if I had not discovered your website.
    Digital Marketing online training

    full stack developer training in pune

    full stack developer training in annanagar

    full stack developer training in tambaram

    full stack developer training in velachery










    ReplyDelete
  6. MounikaSeptember 7, 2018 at 10:37 PM

    Woah this blog is wonderful i like studying your posts. Keep up the great work! You understand, lots of persons are hunting around for this info, you could help them greatly.
    python training in tambaram
    python training in annanagar
    python training in velachery

    ReplyDelete
  7. simbuOctober 3, 2018 at 3:02 AM

    A good blog always comes-up with new and exciting information and while reading I have feel that this blog is really have all those quality that qualify a blog to be a one.I wanted to leave a little comment to support you and wish you a good continuation. Wishing you the best of luck for all your blogging efforts read this.
    java training in chennai | java training in bangalore


    java training in tambaram | java training in velachery

    ReplyDelete
  8. UnknownOctober 8, 2018 at 11:18 PM

    Resources like the one you mentioned here will be very useful to me ! I will post a link to this page on my blog. I am sure my visitors will find that very useful


    angularjs Training in chennai
    angularjs-Training in pune

    angularjs-Training in chennai

    angularjs Training in chennai

    angularjs-Training in tambaram

    ReplyDelete
  9. UnknownOctober 25, 2018 at 4:11 AM

    When I initially commented, I clicked the “Notify me when new comments are added” checkbox and now each time a comment is added I get several emails with the same comment. Is there any way you can remove people from that service? Thanks.

    AWS Interview Questions And Answers

    AWS Training in Chennai | Best AWS Training in Chennai


    AWS Training in Pune | Best Amazon Web Services Training in Pune

    AWS Tutorial |Learn Amazon Web Services Tutorials |AWS Tutorial For Beginners

    ReplyDelete
  10. Sadhana RathoreJanuary 31, 2019 at 4:09 AM

    This is exceedingly helpful information, very good work. Thanks for sharing and let me wait for further updates.
    Machine Learning course in Chennai
    Machine Learning institute in Chennai
    Data Analytics Courses in Chennai
    Big Data Analytics Courses in Chennai
    DevOps course in Chennai
    Best DevOps Training in Chennai
    Machine Learning Training in Anna Nagar
    Machine Learning Training in T Nagar

    ReplyDelete
  11. dhanamMarch 4, 2019 at 3:36 AM

    It was worth visiting your blog and I have bookmarked your blog. Hope to visit again
    Microsoft Azure online training
    Selenium online training
    Java online training
    Java Script online training
    Share Point online training

    ReplyDelete
  12. PadminiprwatechDecember 10, 2019 at 10:21 PM

    Thanks for sharing your innovative ideas to our vision. I have read your blog and I gathered some new information through your blog. Your blog is really very informative and unique. Keep posting like this. Awaiting for your further update. If you are looking for any Python programming related information, please visit our website python training institute in BTM layout

    ReplyDelete
  13. harisalee@64gmail.comMarch 9, 2020 at 3:50 PM

    Such a well and helpful blog..
    Thanks for sharing with us,
    We are again come on your website,
    Thanks and good day,
    If you need any logo then,
    Please visit our site,
    buylogo

    ReplyDelete
  14. varshaMay 13, 2020 at 10:06 PM

    These concept is a good way to enhance the knowledge. I like it and help me to development very well.Thank you for this brief explanation and very nice information.Well, got a good knowledge.
    AWS training in chennai | AWS training in annanagar | AWS training in omr | AWS training in porur | AWS training in tambaram | AWS training in velachery

    ReplyDelete
  15. nishaMay 30, 2020 at 5:46 AM

    The blog is really very informative. every concept of this blog is represent very uniquely. concepts are very clearly explained.

    Data Science Training Course In Chennai | Data Science Training Course In Anna Nagar | Data Science Training Course In OMR | Data Science Training Course In Porur | Data Science Training Course In Tambaram | Data Science Training Course In Velachery

    ReplyDelete
  16. RevathiJuly 16, 2020 at 6:23 AM

    Well somehow I got to read lots of articles on your blog.keep it up!!

    android training in chennai

    android online training in chennai

    android training in bangalore

    android training in hyderabad

    android Training in coimbatore

    android training

    android online training

    ReplyDelete
  17. deviJuly 31, 2020 at 3:17 AM

    This is an awesome blog. Really very informative and creative contents. This concept is a good way to enhance the knowledge. Thanks for sharing.Nice blog,I understood the topic very clearly,And want to study more like this.

    Data Science Training In Chennai

    Data Science Online Training In Chennai

    Data Science Training In Bangalore

    Data Science Training In Hyderabad

    Data Science Training In Coimbatore

    Data Science Training

    Data Science Online Training

    ReplyDelete
  18. jeniSeptember 1, 2020 at 7:13 AM

    Thanks a lot very much for the high your blog post quality and results-oriented help. I won’t think twice to endorse to anybody who wants and needs support about this area.

    oracle training in chennai

    oracle training in velachery

    oracle dba training in chennai

    oracle dba training in velachery

    ccna training in chennai

    ccna training in velachery

    seo training in chennai

    seo training in velachery

    ReplyDelete
  19. Jhone linenApril 24, 2021 at 6:28 AM

    Amazing post thanks for sharing us.
    buy a logo

    ReplyDelete
  20. mirApril 28, 2021 at 12:00 PM

    I was trying to find some content that is about design & now I found it thanks for sharing with us!
    we have years of experience in web designing or logo designing if you are interested in web designing or logo designing then visit us?

    ReplyDelete
  21. UnknownJuly 23, 2021 at 9:25 AM

    instagram takipçi satın al
    instagram takipçi satın al
    aşk kitapları
    tiktok takipçi satın al
    instagram beğeni satın al
    youtube abone satın al
    twitter takipçi satın al
    tiktok beğeni satın al
    tiktok izlenme satın al
    twitter takipçi satın al
    tiktok takipçi satın al
    youtube abone satın al
    tiktok beğeni satın al
    instagram beğeni satın al
    trend topic satın al
    trend topic satın al
    youtube abone satın al
    instagram takipçi satın al
    beğeni satın al
    tiktok izlenme satın al
    sms onay
    youtube izlenme satın al
    tiktok beğeni satın al
    sms onay
    sms onay
    perde modelleri
    instagram takipçi satın al
    takipçi satın al
    tiktok jeton hilesi
    instagram takipçi satın al pubg uc satın al
    sultanbet
    marsbahis
    betboo
    betboo
    betboo

    ReplyDelete
  22. Mrbk30December 22, 2021 at 3:03 AM

    Very Informative blog thank you for sharing. Keep sharing.

    Best software training institute in Chennai. Make your career development the best by learning software courses.

    microsoft azure training institutes in chennai
    Xamarin Training Course in Chennai
    Best Docker Training in Chennai

    ReplyDelete
  23. Ramesh SampangiMarch 4, 2022 at 9:26 AM

    Informative blog and knowledgeable content. Thanks for sharing this awesome blog with us. If you want to learn data science then follow the below link.
    Data Science Classes in Hyderabad

    ReplyDelete
  24. Tiktok LoaderJanuary 23, 2023 at 3:47 AM

    We feel excited to read your valuable post and please always be inspired and keep posting.

    ReplyDelete

Subscribe to: Post Comments (Atom)