close

Вход

Забыли?

вход по аккаунту

?

Torrent base of software distribution by ALICE at RDIG

код для вставкиСкачать
Torrent base of software
distribution by ALICE at RDIG
V.V. Kotlyar (IHEP, Protvino),
E.A. Ryabinkin (RRC-KI, Moscow), I.A. Tkachenko (RRC-KI, Moscow),
G.S. Shabratova (LHEP JINR, Dubna),
A.K. Zarochentsev (St. Petersburg State University, St. Petersburg)
5th International Conference GRID�2012
"Distributed Computing and Grid-technologies in Science and Education“,
16-21 July, 2012 , Dubna, Russia
AliEn & ALICE Grid
• AliEn: �Alice Environment’
– Central Services at CERN
• Task Queue, Job & File Catalog
• Software management (build & deployment)
– VO box site-specific operations
• Job Agent submission
• Software deployment
• Resource monitoring
– Grid Monitoring with MonaALisa
– Data Management
• AliEn File Catalog
• Grid-Enabled XRootD SEs
This talk
Types of Software Deployment
Traditional
AliTorrent
Central
Management:
Central
Management:
-Routine software builds
-Catalogue& store in AliEn
-Additional AliEn torrent
store
-Catalogue, seeder &
tracker
Site SW operation:
-Job request SW from VO box service
-VO box PackMan service pulls SW
-SW deployed on shred area
-WNs read SW from shared area
Shared SW area:
Site SW operation:
-VO box is not involved
-Jobs pull SW from:
•alitorrent.cern.ch speeder
•local peers
•other sides as available
-though typically behind a FW
Resolves:
-Resource bottleneck
-Single point of failure
-Can require active repairs per site
-Bottleneck & single point failures
-Site level maintenance of shared area
Basic Torrent details
File chunks contain hashes of original
file to Provide Data integrity
Tracker: map of seeders:files
Seeders: have & serve file
Leeches: pull & serve file chunks
AliTorrent Details
• Torrent Features
– Distributed Hash Tables
• Decentralized seeder lookup – seeders are trackers
– Peer Exchange
• Local peer information is propagated by seeders
– Local Peer Discovery
• Multicast to discover peers on same network
• ALICE/AliEn Features
– Total software download is 300--‐400MB
– Enabled per site (VO box) with AliEn LDAP flag
name=<CN_NAME>,ou=CE,ou=Services,ou=<SITE>,ou=Sites,o=alice,dc=cern,dc=ch
installMethod=Torrent
AliTorrent as an opportunity
• AliTorrent use in AliEn
– Reduces problems associated with SW deployment
– Simplifies site operations by removing a VO Box
service
*Does not eliminate VO box model from ALICE Grid
* Does eliminate site-specific VO box requirement
• Elimination of site--‐specific VO box allows for
remote use of other Grid resources: exmp->
OSG
AliTorrent
• ALICE torrent-based software distribution
links: (see at http://alitorrent.cern.ch/)
– Available software packages
– Seeder statistics
– Seeder network traffic monitoring
– Stand-alone torrent clients
Towards production operations
• Site policy caution:
– limits can be broad
– has hampered wider
use
• ALICE Experience:
– initial distrust turns
into recognition of
approach’s merit:
Stable running at CERN, 2 T1s and a
few T2s (including RRC-KI and IHEP)
– simplifies site operations
BitTorrent
Summary
• Software deployment on shared area
– Bottleneck & site-level single point failure
– site-level SW corruption requires admin intervention
• Torrent model
AliTorrent
– Removes bottleneck & site-level single point of failure
– Eliminates a site service & reduces site management
– Performance capabilities meets typical ALICE workflow & site
requirements
Eliminates requirement for site‐specific VO box
• AliTorrent is a site-friendly tool for opportunistic (or
general) use
– don’t ask the site to “do” something -> install or manage a service
– ask the site to “not do” something -> Block Torrent use.
Talk has been prepared on the base:
-Report of C. Grigoras at ALICE T1/T2 workshop, Karlsruhe, Germany, January, 2012
-Report of R.J. Porter et al at CHEP 2912, New York, USA, May, 2012
Документ
Категория
Презентации
Просмотров
8
Размер файла
333 Кб
Теги
1/--страниц
Пожаловаться на содержимое документа