Welcome to Netease Cloud Community to learn more about Netease technology product operation experience.
“Knowing things by learning is a brand column created by Netease Yunyidun. The words come from Lun Heng Shi Zhi, written by Han Wang Chong. People have higher and lower abilities. Only by learning can they know the truth of things, and then by wisdom, they will not know without asking questions. “Knowing things by learning” hopes to pass a piece of technical dry goods, trend interpretation, charactersThoughts and precipitation bring you gains, but also hope to open your horizons, achievements are not the same as you. Of course, if you have a good understanding or sharing, you are also welcome to submit a contribution by e-mail (email@example.com).
The following is the text:
Author: Guarav Banga
Recently, I participated in a seminar on ideological leadership composed of more than a dozen CISO (Chief Information Security Officer), and we had Jeffersonian discussions on a series of network security issues. The first question raised at the seminar was how people viewed AI and whether AI was being used. Many of themThe participants said that their machine learning project was currently under way, but also stressed that artificial intelligence technology was not used in the field of network security.
Artificial intelligence is a word that deserves our imagination, and it is also in line with our understanding of human intelligence, Turing tests and science fiction movies. Unfortunately, as I explained at the CISO dinner, there are still some confusions about what artificial intelligence is.Although many people have been talking about AI in recent years, people’s confusion has not been solved.
This paper consists of three parts. Firstly, we discuss some basic concepts of human intelligence and artificial intelligence, and explain some popular vocabulary today, including the differences between artificial intelligence, machine learning, expert system and in-depth learning. Finally, we will discuss the reality of AI in network security applications.And why do we need to use it as a strategic tool?
- What is intelligence?
- Conventional Artificial Intelligence and Narrow Artificial Intelligence
- The Difference between Artificial Intelligence, Machine Learning, Expert System and Deep Learning
- Artificial Neural Network and Deep Learning
- Application of Artificial Intelligence, Machine Learning and Deep Learning in Network Security
- Artificial Intelligence in Network Security
What is intelligence?
Before discussing the topic of artificial intelligence further, let’s first define what intelligence is. Intelligence is quite complex in a broad sense, and there are fierce debates in many aspects of science and philosophy. But in this article, I provideThe following definition is given.
I have two very important ideas about intelligence. First, many scientists believe that human intelligence is rooted in how the brain discovers and stores correlated layered patterns in a variety of different types of sensory data. For example, when you see a “Ga” in a captured packet or log file, there is a “Ga” in a network name.When “urav-iPhone” comes up, it’s natural to think that it’s probably your friend Gaurav’s iPhone. You unconsciously associate knowledge about the names of colleagues with knowledge about common device types. In life, you will constantly unconsciously update these two.These models are also influenced by multi-source multimedia sensory input, including Apple’s advertising, TV shows, e-mail, articles, and conversations in the corridor. You can try to compare this process with the traditional arbitrary string pattern matching program and have flexibility in input.It is consistent with the accuracy of output.
Secondly, intelligence is a kind of prediction, which is a way to solve the problem. For example, your eye is trying to see everything it can see, while the brain sends predictive information to the eye through the nervous system based on what it expects the eye to see. This predictive mechanism “fills in” what it did not realize.Things, that’s why you can’t normally recognize blind spots. This predictive mechanism also allows you to walk in a dark bedroom at night without tripping: your brain sends signals to the motor nervous system, providing the muscles with the desired model for walking.
Conventional Artificial Intelligence and Narrow Artificial Intelligence
The concept of artificial intelligence was first proposed by some computer scientists who explored beyond traditional programs in the 1950s. They were inspired by super-intelligent programs that have intelligent features similar to those of humans, such as R2D2 and C-3PO in Star Wars, and Superm.Supercomputers in an III, which is conventional AI. Conventional AI does not exist today. We don’t know how to imitate the human brain to work, or even a small part of its intelligence.
The AI that exists today can be called narrow AI. Nowadays, there are many useful products using narrow AI. They can perform some tasks with high quality and quantity, even better than human beings. Amazon’s Alexa, for example, has a limited input range, but combinesMany narrow AI technologies accomplish certain tasks, which makes people mistakenly think that it has intelligence. The current world champions of chess and Go are also the application of narrow AI. These narrow AI systems have three intelligent elements discussed earlier: storing domain-specific knowledge and acquiring new knowledge.Mechanisms and mechanisms for using these knowledge.
At present, there are also several methods to solve the problems in the field of network security through narrow AI. Although there are no security robots that can test and replace security team members through Turing, tools based on narrow AI can detect threats and vulnerabilities ahead of time and are better than most people.Measure the security situation.
The Difference between Artificial Intelligence, Machine Learning, Expert System and Deep Learning
Machine learning is the application of inductive algorithm, the first step in the process of knowledge acquisition, and it was generated in the process of exploring artificial intelligence in the 1960s. Machine learning can be said to focus on “learning” algorithm. Computers do not accomplish tasks by writing specific computer instructions, but by using large amounts of data.Practice “training” to enable them to learn how to perform tasks. Samples for training can be provided either externally or in the previous stage of the knowledge discovery process.
Over the years, there have been many machine learning algorithms, including decision tree, inductive logic, clustering, Bayesian network and artificial neural network. Artificial intelligence is closely related to statistics and even overlaps with each other.
Machine learning is considered to be born out of expert system, but unlike it, expert system solves problems by reasoning based on fuzzy rules according to well-prepared knowledge system (rules). Expert systems are touted as the most successful case of AI in the 1980s. The principle behind expert systems is intelligence.Systems can acquire capabilities from the knowledge they have rather than from the specific reasoning strategies they use. In short, expert systems have knowledge, but they are not self-taught. They need human programmers or operators to make them smarter. But if we decide on intelligenceIf righteousness is judged, they are not smart.
Now back to the learning system. Machine learning is difficult because the association pattern between data from multiple dimensions is a difficult problem. This is a big data and computing intensive problem. The human brain continuously acquires a large amount of sensory data from a large number of sources and across multiple dimensions, gradually perfecting its model, and thenOnly in this way can we achieve the level of intelligence and professional knowledge of skilled employees in the network security team. Imagine the amount of training data (labeled and unlabeled) that a graduate’s brain receives. In most cases, the training data suitable for machine learning systems are rather scarce, which makes machine learning programs impossible to mention.For accurate results.
Artificial Neural Network and Deep Learning
In recent years, we have seen a rapid development of machine learning technology called deep learning, which is the evolution of an early machine learning method, artificial neural network, which is inspired by the structure of the human brain. In the neural network, each node assigns weight to its input.Indicates whether the operation it is performing is correct or not. The final output is determined by the sum of these weights. In practice, there are many layers of neural network, each layer corresponds to each sub-task of neural network.
The output of the neural network report is in the form of a “probability vector”. For example, the system may say that the image contains 90% confidence of a given animal, and that the animal has 25% confidence of a crocodile.
Until now, the study of neural networks has produced almost nothing that can be called “intelligence”. The predicted output is of little certainty and therefore of little use. As you may have thought, the most basic problem we are facing now is that even the most basic neural network has a high computational density.It is impractical to establish and use neural networks to accomplish complex tasks. A small research team led by Geoffrey Hinton of the University of Toronto has been working on this issue, and they have demonstrated this idea by parallelizing supercomputers’algorithms.
To understand this problem, let’s take an example of computer vision and autopilot: identify traffic signs. When the parking sign detection neural network is trained, many incorrect answers are likely to appear. For example, it may do well in good visibility, but in bad weather.Not under gas conditions. This network needs a lot of training. It needs to see tens of thousands or even millions of images until the weights of various neuron inputs are adjusted, and it can get the right answer every time regardless of the environmental conditions. Until we do that, we probably won’t talk about this nerve.The Internet has learned how to look like a parking sign.
That’s what Andrew Ng did at Google in 2012. Wu’s major breakthrough was to increase the number of layers and neurons in the neural network, and then train it by running massive data, mainly 10 million from YouTube videos.Image. The “depth” in deep learning indicates that the neural network contains a large number of layers. The Google Brain Project is a neural network trained with in-depth learning algorithms on 16,000 CPU cores. The system has learned to recognize things in YouTube videos, such as “cats,” even thoughThe system has never been told what a cat is. Neural networks can “see” the visual image of a cat, the visual image containing the word “cat”, and the correlation between audio containing the word “cat”, and learn this correlation as knowledge, just like a child.
Today, image recognition based on in-depth learning is usually better than human beings, such as automobile driving, cancer recognition in blood and tumors in magnetic resonance scans. There are also many variants of in-depth learning that have been actively improved and applied. Some models can be stacked to produce more advanced classification capabilities.The following pictures are from the demonstration of Amazon Rekognition System. Deep learning technology can identify objects, faces and contexts in images and video streams.
Is such a system intelligent? Because deep learning and other advanced machine learning algorithms do learn in their respective fields and become quite knowledgeable, they do have two key factors of “intelligence”.
Do such systems know how to use their knowledge to solve problems? Narrow AI systems currently require manual intervention in order to be associated with real world problem solving workflow and interact with traditional systems and other personnel. People need to understand the traffic cameras that will detect trackers and others.Facial detection and image detection systems are integrated and trained using image data from California DMV driver’s license and vehicle license database.
If such a system is installed in public places, we can greatly improve the community’s ability to fight crime. Imagine that this may also have a multiplier effect. Our police, security personnel and investigators have the potential to improve their efficiency and speed every day! If this narrow sense of AI is used as a weapon, thenSome Secret Service personnel will become more efficient.
The relationship between AI, ML, expert system and in-depth learning can also be understood by referring to the following Venn diagram.
Application of Artificial Intelligence, Machine Learning and Deep Learning in Network Security
Network security can be understood as ensuring the confidentiality, availability and integrity of computer systems. Network defense mainly includes the following three aspects:
Set up and manage effective security control.
Handling and responding to security events.
In recent years, network security has become a multi-dimensional problem. With the “computerization” of business, the number and types of vulnerabilities have increased dramatically. Security researchers and hackers find new ways to harm computer systems every day.
Let’s illustrate this at the attack level, such as business applications and shared passwords. The Yahoo or LinkedIn password of an enterprise user may be the same as that of an enterprise application. Therefore, if Yahoo or LinkedIn are attacked and the password is stolen (and no secret key is set),Then you have a problem: hackers have a million ways to get into enterprise applications.
Usually, defenders do not know how the risk of password sharing affects their business. At this point, you’d better hope that your two-factor authentication configuration is correct and effective.
Attackers exploit multiple vulnerabilities to break through your network, then jump to the target system, enhance their access rights, and finally attack, leak or destroy information. For an organization with more than 10,000 people, we estimate that there are more than 100 million time-varying factors at the level of attack.
This is no longer a personal problem. There are too many data to be analyzed manually.
In order to prevent the network from being broken, defenders must find and solve these vulnerabilities in time, which usually involves re-setting or patching the system, user training, installing other security software, and optimizing the process.
Finally, despite our best efforts, the network was broken. The number of security alarms that security administrators need to deal with every day has been increasing. Alarm handling involves getting data from multiple systems, which is rather tedious and time-consuming. Most institutions do not have enough trained personnel to handle eachDays of safety alerts.
Artificial Intelligence in Network Security
Now imagine that if you have a trained self-learning system, it can automatically and continuously collect data about the enterprise from a variety of sources and correlate patterns in hundreds of dimensions. The system includes the following kinds of intelligence:
Understand all the details related to enterprise assets (configuration, usage, etc.), including all devices, users and applications, both internal and external.
Deeply understand the business importance of each asset and user.
Update the latest knowledge of global and industry-specific threats in a timely manner, that is, the latest threats on a daily or weekly basis.
Deeply understand the various security products and processes that have been deployed.
Combine all the information in Items 1-4 above, calculate your effective risk, and predict where and how you are most likely to be attacked.
Provide normative suggestions on how to configure and enhance security control and process, improve network flexibility, but at the same time will not have a negative impact on business operations.
Provide as much reference information as possible for the priority and mode of handling security alerts, and minimize the impact on them as much as possible.
By providing a variety of visual interfaces and reports, all stakeholders, such as users, business owners, security operators, CISO (Information Security Officer), auditors, CIOs, CEOs and board members, are provided with relevant information to explain their predictions and recommendations.
This article is translated by Netease Yunyidun.
Netease cloud security (Yishi) is based on Netease’s 20-year technology accumulation and security data. It provides integrated security solutions such as anti-garbage, authentication code, registration protection, login protection, anti-cheating activities, application reinforcement, DDoS protection, etc. for Internet industries. It provides perfect technical support throughout the process and helps to build products.Safety protection system, click free trial.
【Preventing malicious orders by E-commerce