Flamenco, Computational Cluster Development Story Continued. The End
Here is the last part of the big story about the Flamenco project. We would like to talk about tasks that the system can solve and to give a conclusion of all Flamenco articles.
Examples of tasks
- Initial code compilation
- Audio/video coding
- Scientific tasks: climate forecasting, research on protein folding, search of prime numbers, etc.
- Information security: selection of passwords to ciphered archives, password search corresponding to certain hashes such as MD5, SHA256, etc.
- Forecasting of market indices
- Internet: distributed web crawlers, study of structure and Network topology
Examples of projects
A list of international noncommercial projects using distributed computing on a global basis is as follows:
- LHC @home
- SETI @home
- Folding @home
- MoneyBee
- RainbowCrack
Conclusion
As a consequence of the work done, we have managed to create a steady computing cluster both in terms of topology and processing of nonfunctional failures of subtasks implementation. Peer-to-peer architecture of the cluster has been implemented. This gave an additional possibility of load distribution between nodes, due to their dual nature that can act on a par both as a subtask performer and as a master node. Flexibility of architecture that was obtained to a large extent due to the modular bus OSGi opens wide possibilities for the development of plug-ins implementing tasks of parallel data processing. Development of new applications can often resolve itself into writing of the Java-class implementing a simple interface and being formalized as an OSGi bundle. Additional tools for cluster administration as a whole have been implemented, for example, a node control console and a browser system of the repository contents. The comprehensive documentation describing both architecture of Flamenco and containing examples of plug-in writing has also been developed and supported. Statistical data on Flamenco use has been collected.
The project backlog contains perspective rework such as monitoring and nodes dynamic configuration which will be conducted soon. The theoretical and practical sides of control over super task fulfillment are under consideration. The control is being carried out by means of grouping and statistic weighing of subtasks being part of super-task direct graph. Project ergonomics, safety issues, lightweightness of the project are being developed as well.
All the aforesaid gives us a belief in prospects and actuality of our work, the results of which we have presented in this story.
The end.
Previous parts
Flamenco, Computational Cluster. Part 1: Introduction
Flamenco, Computational Cluster. Part 2: Flamenco. Peer-To-Peer Topology
Flamenco, Computational Cluster Development Story Continued. Part 3
Flamenco, Computational Cluster Development Story Continued. Part 4
