Use the principle of induction to prove that 1
2 + 2
2 + ¡¤ ¡¤ ¡¤ + n
2 = 1/6 n(n + 1)(2n + 1)
It was a fairly standard proof, and Lexus completed it without much trouble. Prove the base case, prove the inductive step, tie it all together.
Let P(n) be the statement that 1
2 + 2
2 + ¡¤ ¡¤ ¡¤ + n
2 = 1/6 n(n + 1)(2n + 1)
Base case: when n = 1,
1
2 = 1
1/6 (1)(2)(3) = 1
Therefore the base case is true.
Inductive step:
Assume P(k) is true for some k ¡Ê ?, i.e. 1
2 + 2
2 + ¡¤ ¡¤ ¡¤ + k
2 = 1/6 k(k + 1)(2k + 1)
Then
1
2 + 2
2 + ¡¤ ¡¤ ¡¤ + k
2 + (k + 1)
2 = 1/6 k(k + 1)(2k + 1) + (k + 1)
2
= 1/6 k(k + 1)(2k + 1) + 1/6 (6k + 6)(k + 1)
= 1/6 (k + 1)(2k
2 + k + 6k + 6)
= 1/6 (k + 1)(2k
2 + 7k + 6)
= 1/6 (k + 1)(k + 2)(2k + 3)
Hence P(k) is true ? P(k+1) is true.
By induction, 1
2 + 2
2 + ¡¤ ¡¤ ¡¤ + n
2 = 1/6 n(n + 1)(2n + 1) for all positive integers n.
---
Correct
---
The last thing Lexus wanted to check was whether the system could tell whether answers from other people were right or wrong. This was easy to check: Lexus simply flipped to a random page on "Discrete Mathematics", looked at one of the worked examples, and waited for the system to do something.
---
Correct
---
Lexus was amazed. "System, can you read my mind?"
"No, at least not at this stage. It''s obvious enough what you want me to do though, especially when you''re just sitting there like an idiot, doing nothing but stare at the symbols," the system replied.
Lexus was in a good mood at the moment, so he decided to forgive the system''s snarky antics. "System, can you leave this feature on every time I work on a question?"
"Yes, that is possible," the system replied.
Lexus tested the feature on a few more questions, working on a problem sheet for his digital electronics module due a few days later. He hadn''t yet come across a question in which the feature didn''t work. Be it calculations, proofs, or even diagrams, as long as there was a right answer and a wrong answer, the system could tell him whether his work was right or wrong.
There were a few problems on the problem sheet that the system told him was wrong, and initially Lexus thought the system was talking crap. However, in every case it only took a few seconds of looking over the working closely to discover the mistakes, and after a few of these experiences Lexus came to trust the system more than his own intuition.
Lexus was starting to realise how overpowered this feature was. It only gave him one piece of information, a single word, but it meant so much more. He no longer had to hope for the best and instead could be certain about his work. Well, perhaps he would be certain that he was wrong, but he would rather know the cold hard truth than live in blissful ignorance.
At his current stage though, he couldn''t do much more with the feature than to double check his work when attempting textbook exercises or problem sheets.
And it also didn''t help if Lexus couldn''t figure out how to do the question. He didn''t need the system''s notifications to know that he was wrong, and no amount of "this is wrong" would get him any closer to solving the actual problem. Lexus knew full well that this wasn''t something a level 1 feature could possible do though, so he wasn''t bummed out at the fact that this pipe dream didn''t come true.
Having tested the new feature to his heart''s content, Lexus returned to continuing with his work, completing his problem sheets and programming tasks for university before moving on to make some progress with the reading quest.
=====
"System, status!"
---
Name: Lexus Kagan
Current Quest: Background Knowledge [6/10]
Computer Science: Level 0 [EXP 30/100 (30%)]
Mathematics: Level 1 [EXP 94/1000 (9%)]
Physics: Level 0 [EXP 0/100 (0%)]
Engineering: Level 0 [EXP 0/100 (0%)]
Chemistry: Level 0 [EXP 0/100 (0%)]
Biology: Level 0 [EXP 0/100 (0%)]
---
Lexus looked at his screen, and sighed.
It had been over two weeks since his mathematics levelled up, and Lexus had made some progress with the quest, completing the 6 mathematics textbooks on the quest list. He hadn''t expected the required experience to multiply directly to 1000 EXP points though. Seriously. 1000! It had taken him forever just to level up from 0 to 1, and at the rate he was going, he wasn''t going to level up mathematics again any time soon.
On the other hand, his computer science progress looked more promising.
Lexus pulled out the list of 10 books that the system had tasked him to read, and made a note of the remaining books he had to read:
1. Cormen, T.H., Leiserson, C.D., Rivest, R.L. and Stein, C. (2009). Introduction to Algorithms
2. Kleinberg, J. and Tardos, ¨¦. (2006). Algorithm design
3. Sipser, M. (2012). Introduction to the Theory of Computation
4. Russell, S.J. and Norvig, P. (2010). Artificial Intelligence: A Modern Approach
The remaining books, according to Lexus'' understanding, were all directly related to computer science, so Lexus expected to start gaining computer science experience soon. His current 30 experience points in computer science were entirely from the quest bonuses, which was why his mathematics level was so far ahead of his computer science level.
However, that was going to change soon.
"System, I''m going to get computer science experience out of these right?" Lexus asked.
"Mostly, yes," the system replied.
"Great!"
Lexus was excited. He didn''t know what he would do if the system said "no", because 4 books wasn''t nearly enough to push his mathematics level from 1 to 2. However, it was more than enough to level up his computer science, so Lexus was very motivated to finish the quest once and for all. Off to the college library it was!
Chapter 5 - The Duty of a Computer Science Student
What is the duty of a computer science student?
If you ask the system, it would say "the duty of a computer science student is, of course, to study computer science."
GPT3, a language artificial intelligence model, would agree. If you ask it the same question, it would reply, "the duty of a computer science student is to study the science of computers. This includes the hardware, software, and networks that make up a computer system."
Even a human like Arthur would say the same thing, perhaps adding that "a computer science student should learn to solve problems using computers."
Yes, to study computer science was the core duty of a computer science student.
"So why on earth have you taken a month and a half to even start learning computer science?" asked the system.
Lexus looked up from the current book that he was reading, "Introduction to Algorithms".
He was sitting in his usual seat at the library - tucked away in a corner, at the end of the long wooden table. This didn''t mean that there wasn''t anyone nearby though. From the corner of his eye, Lexus could count at least three others sat comfortably within hearing distance. All of them were staring at their laptop screens, and one or two of the even had their headphones on, but Lexus didn''t want to take the risk of someone overhearing and so decided not to reply to the system''s remark.
Verbally, that is. He simply stuck his tongue out, then continued with his work as if nothing had happened.
To Lexus, learning mathematics first seemed like the obvious choice. After reading his first book, "How to think like a mathematician", and discovering that the book he read determined the type of experience point he obtained, he decided to level up as quickly as possible by focusing his efforts on a single subject. Min-maxing is the way to go, he thought.
Now that he had exhausted all of the mathematics textbooks on the quest list the system gave him, he could refocus on learning computer science.
He was finally in his zone, like a fish that had to struggle on the edge of the beach to finally roll itself back into the water. It felt great to be learning things he was truly interested in.
"Introduction to Algorithms" was a book that he owned, unlike any of the previous books where he had to borrow them from the library. He had previously bought the book while in high school, but the looming threat of exams and university admissions meant that he never really had the time to sit down and read through the book in peace and quiet.
Now that he had this opportunity, he wasn''t going to let anyone disturb him from reading this book, not even the system.
Poor timing for the system to start bugging him again. "Cat got your tongue?" The system sounded very proud of itself for learning how to use idioms.
"Shhhh, be quiet," Lexus whispered. "This is a library."
The system snorted. "You''re the only one that can hear me."
"Yeah, but you''re not the only one that can hear me."
"Then you can just keep quiet and listen to me talk," the system replied.
Why was Lexus even wasting his time trying to talk sense into the system? He shook his head and forced himself to focus, determined to completely ignore external distractions, which happened to include a voice inside his brain.
=====
"I knew you would be here." Arthur walked up to the table where Lexus was and sat down at the adjacent seat.
Lexus looked up from his work, and smiled. "Heya."
"Playing favourites, much?" the system muttered. Lexus ignored it.
Arthur leaned over and glanced at the textbook that Lexus was studying. "Ooh, CLRS. The cr¨¨me de la cr¨¨me of CS textbooks."
"Why, have you read it?" Lexus asked.
"I''ve skimmed a few chapters here and there, but not cover to cover. It doesn''t really help much for competitive programming. Too much math, proofs and niche algorithms that aren''t that helpful," Arthur replied.
Arthur loved competitive programming. He joined all of the programming competitions in high school, being a Codeforces grandmaster and IOI silver medalist. It was his go-to hobby, and he spent pretty much every waking minute doing it. Lexus was always astonished at the persistence and vigour with which Arthur pursued competitive programming, and admired him for it.
Lexus, on the other hand, never really delved deep into competitive programming. He had tried it out a couple of times, but never really saw the point and so gave up early. To Lexus, he enjoyed learning about the algorithms and the proofs behind them, but training for competitive programming felt a bit too much like practicing for an exam and not learning for the sake of learning. At the end of the day, Lexus didn''t find much enjoyment in competitive programming, so there was no point training for it.
"Looks like a good book for me then," Lexus said.
Arthur chuckled. "Yeah, sounds like something you''ll like."
Arthur settled down into the seat next to Lexus, pulled out his laptop and soon started typing away, presumably at one of his coding projects. Lexus also returned to reading "Introduction to Algorithms", and the atmosphere returned to silence.
=====
Lexus was taking an algorithms course this term, so much of the initial chapters of the book wasn''t new to him. However, he made sure to learn the content thoroughly, completing all of the exercises and reading through each sentence carefully.
He knew that this wasn''t the best way to earn as much experience as possible in the shortest amount of time, but he didn''t care. Lexus was enjoying the process of thinking through the exercises, coming up with code, and proving that his algorithms were correct.
This story originates from Royal Road. Ensure the author gets the support they deserve by reading it there.
Consider the searching problem:
Input: A sequence of n numbers A =
1, a2,..., an> and a value v.
Output: An index i such that v = A[i] or the special value NIL if v does not appear in A.
Write code for linear search, which scans through the sequence, looking for v. Using a loop invariant, prove that your algorithm is correct.
It was a simple question meant to teach the principles of loop invariants, but Lexus completed it anyways. It served as good revision for the course, and it wouldn''t take up a lot of time, so why not?
Lexus took out his laptop, and typed:
int LinearSearch(vector A, int v) {
for (int i=0; i-1(b) when b ¡Ê Rf and g(b) = a when b ? Rf
where a is some element in A and b is any element in B. Then g is a surjective function, therefore the proposition is true.
"Good," said the professor. "What does this proposition then tell you about the relationship between A and P(A)?"
Lexus looked back at the question, trying to reverse engineer his way to the solution. The question was asking him to prove that there is no injection from P(A) to A, so by the proposition that he just proved, it would mean that he would need to show that there was no surjection from A to P(A).
Which, now that he thought about it, he always knew from Cantor''s Theorem.
Ding!
---
Mathematics EXP +1
---
Lexus wrote down his thought process, but this time in reverse order so that it made sense logically. Going from Cantor''s Theorem, proving the proposition, then showing that an injection did not exist, as if it did, it would contradict Cantor''s Theorem.
Lexus noted the tiny little screen that only he could see:
---
Correct
---
Proof complete.
Chapter 6 - Strongly Connected Components
As the days passed by, Lexus slowly worked through the chapters of "Introduction to Algorithms", learning interesting concepts like randomised algorithms, quick sort, hash tables, minimum spanning trees and much more. Each chapter felt like a whole new world to explore, full of mysteries and perplexities.
Lexus was having fun treasure hunting in the forest of knowledge: finding rare items, equipping new tools, fighting strong monsters and eventually leaving the forest stronger than before. He went in wearing nothing more than beginner armour, his hand holding a shoddy dagger, and walked out with a bulletproof Kevlar vest over his chest and an assault rifle strapped on his back.
Oh, except the fact that trying to understand something was bloody hard. Lexus had to grit his teeth and grapple with each and every mathematical monster, going through the book line by line, searching up the things he didn''t understand. When that didn''t work, he would try to program the concept using C++.
C++ wasn''t the programming language he was the most familiar with. Like everyone else, he started off his programming journey with Python, known for its resemblance to natural language. But he had wanted to learn C++ for a long time, and now was as good a time as any.
Ding!
---
Computer Science EXP +1
---
And he discovered that he would earn more experience points, even though it increased his pain ever so slightly.
By slightly, he meant a lot.
He was too ashamed to ever admit it to anyone, but he spent over an hour just trying to get the merge sort code to work. Was it the vector initialisation in line 20 that was the problem? Or did he pass in the wrong parameters to the MergeSort() method? By the end, he had over ten print statements littered all over his code in an attempt to catch the error. And he did catch it eventually - it was an off-by-one error.
As the famous saying goes, "There are two hard things in computer science: cache invalidation, naming things, and off-by-one errors."
Frustrated, Lexus leaned back on his chair. When he looked up, he could see the multitude of books that filled his shelf. A few were textbooks that he had borrowed from the library, but there were also books that he had brought with him to university. Popular science books about physics, maths and computer science, as well as biographies about famous scientists like Alan Turing and Albert Einstein.
With all their intelligence, did they ever experience the feeling of being stuck and stupid?
¡°Feeling stuck?¡± said the voice in his head.
¡°Hey system, am I an idiot?¡± Lexus asked into the empty room.
The system snorted. ¡°You¡¯re only realising that now?¡±
¡°Tch.¡± Lexus suddenly felt more determined, if only to prove the system wrong. He was going to make it big!
In order to get through the thousand-page book without going insane, he needed a change of strategy.
It was time to use his ¡°phone-a-friend¡± lifeline. Except he didn¡¯t have to literally phone them, since that friend was right down the corridor.
So Lexus got up from his chair and walked over to Arthur''s room. Knock. Knock knock.
"Come in," a voice called.
Lexus pushed open the door. The first thing he noticed was the sound of chants, beats and hype music that originated from somewhere to his left. He looked over to see Arthur reclined on his study chair with his attention fully focused on his switch, fighting a Pok¨¦mon gym battle.
Without even looking up to acknowledge Lexus'' presence, Arthur spoke.
"Hold on, Duraludon''s the last Pok¨¦mon I have to beat," Arthur said.
Lexus walked over and leaned against the corner of Arthur¡¯s table. "How did you know it was me?"
Arthur replied, "Who else would come knocking at my door at this time of the day? Do you even know what time it is?"
Lexus took a moment to ponder before coming to the conclusion that no, he didn''t know what time it was.
"What time is it?" Lexus asked hesitantly.
Arthur stopped. He placed his switch down on the desk, picked up his phone and shoved the screen in Lexus'' face. "Look. Look at this."
A glaring "00:39" appeared before Lexus'' eyes.
The first thought that came to his mind wasn''t oh no it''s so late I should apologise, but instead it was, "Oh, it''s not as bad as I expected!"
The moment those words came out of his mouth, Lexus knew that he had made a mistake.
"I''m sorry," he quickly added.
Too little too late.
"Well, my night is officially taken up by Pok¨¦mon, so you can come and find me tomorrow morning," Arthur said. "Unless it''s urgent?"
"No, it''s fine," Lexus replied. He had wanted to ask Arthur some questions about a few of the algorithms he had come across in the book, but it could wait. Disturbing Arthur while he was gaming was typically not a good idea, and this rule was doubly true past midnight.
Lexus shut the door as carefully as he could and scampered back to his room, only now noticing how eerily quiet the corridor was.
=====
Lexus woke up at 8am in the morning, but opted to wait another two hours before heading over to Arthur''s room. Arthur was most definitely not an early bird, and 10am was just about the earliest acceptable time if there wasn''t a 9am lecture that morning.
Just as he expected, Arthur had just woken up.
"You''re always so freaking early," Arthur grumbled.
Lexus watched as Arthur climbed out of bed and started getting ready.
Ensure your favorite authors get the support they deserve. Read this novel on Royal Road.
"I''ve already had breakfast, a morning run and spent an hour doing the breadth-first-search tick," he said.
Finding the later chapters of "Introduction to Algorithms" too strenuous to do right after waking up, Lexus chose instead to do something he did know how to do: the optional programming exercise set for his Algorithms 2 course.
"Why am I even wasting my time trying to reason with this machine?" Arthur mumbled.
=====
Arthur sat down on his study chair and swivelled it to face Lexus, who sat down on a stool he grabbed from the other side of the room.
"So, what''s the question that you wanted to ask me last night?" Arthur asked.
"How did you kn- never mind," Lexus said. "I don''t really understand the concept of strongly connected components and the algorithm to find them."
Lexus could tell that Arthur was interested the moment the words "strongly connected components" was said.
"Oh! Kosaraju''s algorithm. It''s a good one, this one," Arthur said as he leaned over to the other side of the table in order to grab a sheet of paper and a pen. "I''ve used it in multiple competitions."
Lexus watched as he started writing on the piece of paper.
"So let''s say we have a di-graph," Arthur said as he drew a couple of circles on the page and randomly added arrows from one circle to another.
"Then a strongly connected component is one where every node is reachable from another node - like for example if we have this triangle over here in the middle, you can go from any one of these three nodes to any other node while following the arrows," Arthur explained.
Lexus nodded and said, "So if you want to get from node number 1 to node number 3 in the directed graph, you can just go through node 2. But 4 isn¡¯t in the same strongly connected component as 1, because you can get from 4 to 1 but you can¡¯t get from 1 to 4."
"Exactly," Arthur replied. "Then Kosaraju''s algorithm makes use of the fact that the transpose graph of any di-graph, basically a graph with all the arrows pointing the other way, have the same strongly connected components as the original graph. If I draw the transpose graph of the one we already have-"
"- the triangle of nodes in the middle are still strongly connected, and so we can find the strongly connected components by exploiting this property."
Lexus made a note to himself to attempt to prove that statement when he got back. It was pretty intuitive, but could he prove it from first principles? Before he could get any further along that line of thinking, Arthur''s voice brought his focus back to reality.
"The procedure for Kosaraju''s algorithm is: first, you perform a depth-first-search of the entire graph, storing the nodes in the order for which they run out of output edges in a stack, then you transpose the graph, then you do another depth-first-search for every node by popping the stack."
"Wh-what." Lexus said. He had no idea what was going on.
Arthur paused, as if to think, and Lexus looked at Arthur incredulously. Did this guy just think he would listen to that and go "Oh that makes total sense, I get it now"?
To be fair, he''d done that before in Professor Emerson''s supervision.
Arthur contemplated for a while more before he said, "I''ve always just used the algorithm without thinking too much about it... maybe it''s better if I just show it to you."
And so Arthur led Lexus through the procedure for Kosaraju''s algorithm step by step, showing him which elements would be added to the stack in which order. By observing the algorithm as it solved the problem, Lexus was able to have a better grasp of what exactly was going on in each step of the way.
1st DFS: 0 ¡ú 1 ¡ú 2 ¡ú 3 then 4 ¡ú 5
stack: 3 2 1 0 5 4
2nd round of DFS:
DFS(4) ¡ú 4
DFS(5) ¡ú 5
DFS(0) ¡ú 0
DFS(1) ¡ú 1 2 3
Therefore there are 4 strongly connected components: {4}, {5}, {0} and {1, 2, 3}.
---
Computer Science EXP +1
---
As he listened to Arthur''s explanation, Lexus was also able to finally figure out why the algorithm worked.
"Oh, I think I get it! The first depth-first-search gives the order of the starting nodes for the second round of depth-first-searches! This makes sure that in the second round, searching from a node would only return the nodes in the same strongly connected component," Lexus said.
---
Computer Science EXP +2
---
"Yeah okay, I guess that kind of makes sense," Arthur replied. Lexus could tell that Arthur didn''t really care about the reason why the algorithm worked, it was enough for him to know that it did what it was meant to do.
But to Lexus, there was a huge difference between knowing and understanding. It was akin to knowing how to drive a car versus knowing how to build a car, the latter was vastly more interesting than the former.
Hmmm. Was that why the system picked him?
Lexus didn''t know, and he probably wouldn''t until much later. For now, all he had to do was focus on learning, gaining experience points, and levelling up. With the help of the system, Lexus was sure that he would be able to go very far in his academics. All it took was some effort, and a little bit of time.
Maybe, just maybe, one day he would be mentioned in textbooks and classrooms, just like many of the great scientists he admired?
Nah. The likes of Isaac Newton and Alan Turing couldn¡¯t be reached just by having a system. But he was curious to see just how far he could go.
=====
For the next month or so, Lexus focused primarily on reading through "Introduction to Algorithms". He had also started "Algorithm Design" in parallel, reading the chapters relating to the same concept. It gave him a new perspective on the same topic, allowing him to form a better understanding of the algorithms involved.
It was also simply much more efficient.
Lexus would ask Arthur about problems he had with implementing the algorithms, but when it came to the mathematics behind them, Arthur wasn''t able to help, so Lexus eventually turned to Professor Emerson for guidance.
Having specialised in theoretical computer science, Professor Emerson had extensive knowledge about many of the algorithms covered in the book, including the mathematics behind it. Initially, Lexus¡¯ questions were limited to the few minutes at the end of every supervision, but eventually the questions piled up so much that the professor set up another weekly meeting with Lexus just so Lexus could ask all his questions at once. The meetings were so enlightening that Lexus gained at least 5 experience points each and every time.
Granted, his head felt like it was about to explode from all the new things that he learnt, but that was a small price to pay.
By the time he had completed the two books, Lexus had accumulated so much experience that he was already close to a level up.
"System, can you show me my status?" Lexus asked.
---
Name: Lexus Kagan
Current Quest: Background Knowledge [8/10]
Computer Science: Level 0 [EXP 94/100 (94%)]
Mathematics: Level 1 [EXP 115/1000 (11%)]
Physics: Level 0 [EXP 0/100 (0%)]
Engineering: Level 0 [EXP 0/100 (0%)]
Chemistry: Level 0 [EXP 0/100 (0%)]
Biology: Level 0 [EXP 0/100 (0%)]
---
Yes, 6 points to go!
"Introduction to Algorithms" and "Algorithm Design" brought him 64 experience points - more than 3 mathematics textbooks combined. It might have been due to the sheer length of the books, but it also probably had to do with how invested he was in the learning process, and how much he actually took out of the words he read.
If he was honest, most of the experience probably came from the weekly meetings with Professor Emerson. It just went to show how much of a difference a good teacher could make. Damn, he should have started asking questions earlier!
Chapter 7 - Levels, Levels
Similar to the previous occasion when his mathematics was close to levelling up, Lexus was studying intently in the privacy of his room. He had found himself gradually making fewer and fewer visits to the library, preferring instead to be able to speak freely to the system whenever he wanted to.
Ding!
---
Computer Science EXP +1
Computer Science levelled up from Level 0 to Level 1. Congratulations!
---
Unlike last time when his mathematics levelled up, Lexus suppressed his excitement, waiting patiently for the next notification.
Ding!
---
Feature unlocked [Computer Science]: Increased Processing Speed (passive)
---
¡°Oh. My. God.¡± Lexus was certain that he had hit the jackpot. ¡°System, does this mean what I think it means?¡±
The system was amused by Lexus¡¯ response. ¡°What do you think it means?¡±
Mesmerised by the words on the notification screen, Lexus reached his hand out in an attempt to touch it. Unsurprisingly, his hand simply passed through without any resistance.
¡°Wait a moment, how do I know whether the feature is actually affecting my learning speed?¡± he asked. He certainly didn¡¯t feel any smarter as a result of the level up.
¡°Get ready for what¡¯s coming,¡± the system said.
¡°Wha-aaaaaaarrrgh!!¡± Lexus screamed and collapsed onto his knees on the floor of his room, his hands clutching his head in an attempt to reduce the searing pain, if even by a little. He was suddenly reminded of the day the system intruded into his brain - it felt like that, but ten times worse. At that moment, Lexus felt like he was at death¡¯s door.
Then, as suddenly as it came, it was gone.
All that was left was the sounds of harsh breaths echoing throughout the room. At this point, Lexus was sprawled on the floor facing up, with one hand over his eyes and sweat covering his entire body. ¡°System, never do that again. Especially without a warning. Please.¡±
The system was sympathetic in its reply. ¡°Sorry, Lexus. Refactoring the neuron synapses in your brain necessitated the pain. Look on the bright side! At least you can learn a bit quicker now, even though you¡¯re still just as dumb as you were before.¡±
At that moment though, Lexus had no capacity to feel the slightest bit of elation. All he could think was, it better be worth it¡
=====
Thanks to the newly obtained feature [Increased Processing Speed], Lexus was able to speed through the rest of the books on the book list in a few short weeks. Despite not having gotten any smarter, being faster allowed Lexus to learn at an unprecedented pace. He devoured the rest of the books like a madman, and soon enough, he received the notification which he had been working towards for months.
---
Congratulations on completing book ¡°Artificial Intelligence: A Modern Approach¡±
Quest progress: 10/10
Mathematics EXP +5, Computer Science EXP +5
: Quest complete!
---
¡°Finally!¡± Lexus breathed out a huge sigh and leaned back on his chair.
Over to his side were the stack of books that he had spent week after week reading: ¡°How to think like a mathematician¡±, ¡°Mathematics for computer science¡±, ¡°A First course in probability¡±, ¡°Linear Algebra¡±, ¡°Discrete mathematics¡±, ¡°Game Theory¡±, ¡°Introduction to Algorithms¡±, ¡°Algorithm design¡±, and finally ¡°Introduction to the Theory of Computation¡±. Lexus picked up the newly read ¡°Artificial Intelligence: A Modern Approach¡± and placed it at the top of the stack.
Looking at the pile of books that he had spent so much time and effort reading, a sense of accomplishment welled up from within. Yes, it was difficult, but it was also interesting.
Enjoying the story? Show your support by reading it on the official site.
After this experience, Lexus was sure that he was going to continue reading and learning, even when there wasn¡¯t a quest driving him. The knowledge and experience were too good to pass up!
When he compared him now to the person he was at the beginning of the year, Lexus couldn¡¯t help but reflect on how far he had come. Outwardly, it seemed like nothing had changed: Lexus was still a first year computer science student, occasionally going out with friends, playing badminton on the side and otherwise spending his time studying his subject.
However, only he knew how much progress he had made in the past few months. He not only knew so much more, he was also more willing to search for knowledge on his own. No longer did he rely on his professors to spoon feed the knowledge to him, he took his own learning by the reins, searching for answers on his own and proactively asking questions to his professors when he needed guidance.
And of course, there was the system. Honestly, he didn¡¯t really know what to think of it when it first came into his life, and he still didn¡¯t.
What was the system? What did it want? Why him, of all people?
His line of thought was interrupted by another notification.
Ding!
---
Quest: Publish a research paper
Embark on your research journey! Publish a paper in a journal or conference with SJR of 3 or above.
Quest Rewards: Mathematics EXP +50*SJR, Computer Science Experience +50*SJR
---
¡°Oh, research!¡± A pang of excitement washed over him. He was glad that the system¡¯s quests aligned well with his career plans; if the system had asked him to join a competitive programming competition, Lexus didn¡¯t know what he would do.
Now that he had been given a research quest, Lexus felt more motivated than ever.
The sudden notification from the system shed some light on his previous queries. ¡°System, is this why you chose me? Because I want to do research?¡± Lexus asked.
The system replied, ¡°Partly, yes. It helps that you¡¯re theoretically inclined, so I don¡¯t have to waste the energy to force you into a research career.¡±
The words ¡°force you into a research career¡± rubbed Lexus the wrong way. ¡°What, if I didn¡¯t want to do research, you would just force me into it? How would you even do that? Can you stop me?¡±
The system didn¡¯t directly answer his question, instead asking, ¡°What would you do if you couldn¡¯t earn experience for the rest of your life?¡±
Lexus paused. His heart clenched. He looked up and around, trying to get a glimpse of the supernatural being that was so rudely intruding in his life, but to no avail. ¡°No. No, you wouldn¡¯t.¡±
The system didn¡¯t expect Lexus to take it so seriously. In an attempt to remedy the situation, it replied, ¡°You¡¯re right, I wouldn¡¯t. I¡¯m stuck with you, so I¡¯ll support you however I can. Trust me, my task is to help you as much as I can.¡±
Despite the system¡¯s assurances, Lexus wasn¡¯t fully convinced. Sure, his life had only gotten better since it appeared. Sure, it had given him a huge boost in terms of academics, and in terms of confidence. Sure, it had bestowed upon him overpowered powers like [Increased Processing Speed].
But how was he to know whether the system meant any harm?
¡°System, can you be quiet for a second? I need to think.¡± Lexus moved over to his bed and crashed on top of the blanket, wrapping his arms tightly around the pillow.
Lexus had always known it, but it was the first time in which he acutely realised that the system wasn¡¯t always his ally. It had an agenda of its own, and would do anything in order to accomplish its goals. The most frustrating thing was, Lexus had no clue as to what its goals were. World domination? Slavery of the human race? Or something more benign, like helping humanity combat its biggest problems?
He was stuck between a rock and a hard place. There was no way Lexus would be able to tell anyone about the system - ¡°oh I happen to have a talking voice inside my head and can see weird glowing screens that give me experience points when I complete a quest¡±? Haha.
But there was also no way that Lexus could blindly obey the system¡¯s instructions without knowing more about its intentions, and what it truly was. He didn¡¯t want to risk it. If he trusted the system too much and followed its instructions blindly, he could potentially cause irreparable damage. He would have to do this covertly though - the system was outrightly blackmailing him, and he didn¡¯t want to throw away his future just yet.
He also didn¡¯t know what would happen if he blatantly ignored the system¡¯s instructions. Given his previous experiences with the searing headaches, it was safe to say that the system had some control over the inner workings of his brain. If he went against the system¡¯s wishes, would his life be at risk? A shiver ran down Lexus¡¯ spine. God, how did I get from worrying about my grades to worrying about my life?
Lexus thought for a while more, and came to the conclusion that there was only one sensible course of action to take.
=====
¡°You want to do research in artificial intelligence?¡± Professor Emerson asked.
Lexus nodded firmly. ¡°Yes. Do you have any advice for how I can get into AI research?¡±
Professor Emerson nodded. ¡°It¡¯s a shame that you¡¯re not into theoretical computer science. Given your potential and work ethic, I¡¯m sure that you¡¯ll be well suited to it. Oh well, do you know which subfield in AI you want to work in? The popularity of AI has exploded in recent years, and we have multiple professors here at Cambridge working in different subfields.¡±
Lexus took a second to consider. Honestly, he didn¡¯t even know if the system counted as a form of artificial intelligence. If it did, it seemed rather¡ advanced. How should he go about trying to understand the entity within his brain?
Seeing Lexus hesitate, the professor said, ¡°What you choose right now isn¡¯t really that important. You¡¯re probably going to switch research topics as your career progresses, so there¡¯s no point worrying about it at the undergraduate level.¡±
Makes sense, Lexus thought. At the stage he was currently at, he wasn¡¯t going to get even remotely close to understanding the truth behind the system, even if the system was understandable using the concepts of artificial intelligence.
¡°Just choose whatever interests you the most,¡± the system said in his head. It obviously had no idea just how much trouble it was causing him. Lexus wanted to scream, ¡°You¡¯re the only reason why I¡¯m having so much trouble deciding what to research!¡± But he refrained.
He didn¡¯t want to look like a madman in front of his professor.
The system and the professor had a point. At the end of the day, his research career was his own, and he should take into account his own interests in addition to his considerations regarding the system. After all, he wouldn¡¯t want to be stuck with a field he didn¡¯t enjoy, having to slog day after day without the promise of any reward. That would be the worst!
At last, Lexus came to a decision. He looked straight into the professor¡¯s eyes, and spoke. ¡°I want to¡¡±
Chapter 8 - Natural Language Processing
¡°I want to do NLP - natural language processing,¡± Lexus said.
As he voiced out his decision, a strong breeze of cold wind blew upon them, causing Lexus¡¯ skin to tingle. Or was it the excitement? He didn¡¯t know.
When he was prompted to do what interests him by the system and the professor, his mind immediately went to natural language processing, which is a subfield of artificial intelligence that aims to allow machines to understand human language.
The authors of ¡°Artificial Intelligence: A Modern Approach¡± wrote extensively about natural language processing in terms of techniques and applications, and Lexus was so intrigued by the topic that he even tried his hand at creating his own language model to generate text from a given prompt. Obviously, it wasn¡¯t even close to being state-of-the-art, but it incited Lexus¡¯ excitement in the topic, making him want to learn even more. It seemed like a field that was making rapid progress, and Lexus wanted to be a part of that action.
And it¡¯ll be helpful, since the only way I can currently interact with the system is through language, Lexus reasoned with himself. If he could learn more about how language models behaved, maybe he would be able to determine if the system was truly an AI in the conventional sense. If it wasn¡¯t¡ was it an entity from outside of this world?
In response to Lexus¡¯ declaration, the system said, ¡°Great choice!¡±
Lexus was weirded out by its overly-nice reaction. What¡¯s wrong with this thing?
Professor Emerson replied, ¡°In that case you should contact Professor Milton, she specialises in that area. I¡¯ll make a recommendation for you, but it¡¯ll be up to you to convince her to take you in.¡±
That was to be expected. Professor Emerson was impressed by the progress that Lexus had made over the past few months, but if Lexus wanted to do research under another professor, she would have to willingly accept him into her team. And it was Lexus¡¯ job to convince her to do so.
=====
¡°Mmmph-¡± Arthur waited until the rice was fully down his throat before speaking. ¡°You¡¯re going to do research under Professor Milton?¡±
¡°Yeah¡±, Lexus replied. ¡°With Professor Emerson¡¯s recommendation, I managed to get into contact with Professor Milton and arranged a meeting with her three days later at the computer lab.¡±
¡°Nice! Mate, I know you¡¯ll smash it. What kind of research are you gonna do?¡± Arthur asked in between mouthfuls of rice.
Lexus laughed at Arthur¡¯s enthusiasm. ¡°Haha, why are you more excited than I am?¡±
Arthur put his spoon down with a clang, stating, ¡°Of course! My mate¡¯s going to be the best scientist in the world. When are you gonna win the Turing award?¡±
The system chipped in, ¡°Look, someone who knows what they¡¯re talking about.¡±
¡°I haven¡¯t even started doing research yet, you¡¯re thinking too far ahead,¡± Lexus said. ¡°You know about natural language processing?¡±
Arthur replied, ¡°I don¡¯t know a lot about it, but wasn¡¯t there this craze about large language models a few years ago? Everyone seemed to be going on about it.¡±
Lexus took a sip of water. ¡°Yeah, you¡¯re probably talking about the time when GPT-3 was introduced by OpenAI in 2020. Did you know of the time when a student at Berkeley used it to write a series of self help blog posts and published them as an experiment? He didn¡¯t even mention that it was written by an AI, and one of the blog posts made it to the top of Hacker News.¡±
¡°Damn, that¡¯s crazy. Do you think it passes the Turing test?¡± Arthur asked.
¡°I¡¯m not sure, but it¡¯s probably close. If people didn¡¯t realise that the posts were written by an AI, then the text that it generates shouldn¡¯t be too far of from typical human text,¡± Lexus replied.
¡°Man, twitter was going off about how GPT-3 was going to take over the world. That, and the funny responses that it would output from time to time. Hold on, there was one about chess that was pretty funny,¡± Arthur said.
¡°Yeah, the guy from Berkeley probably had do do some prompt engineering,¡± Lexus replied.
Arthur took out his phone and started searching for something on the app. After a while, he said, ¡°This! This is the one I was talking about.¡±
Arthur showed Lexus the tweet, which went like this:
Human: Do you want to play chess?
AI: I love playing chess!
Human: Okay, you start.
AI: I start with the white pieces.
Human: What¡¯s your opening move?
AI: My opening move is to place the white king¡¯s pawn two squares forward.
Human: I¡¯ll match that with black king¡¯s pawn two squares forward.
AI: Excellent move!
Human: Your turn.
AI: I move my white king¡¯s pawn two squares forward.
Human: That¡¯s not allowed.
AI: Oh, I¡¯m sorry. I didn¡¯t know that.
Human: Try again.
AI: I move my white king¡¯s pawn two squares forward.
Human: I just told you that¡¯s not allowed.
AI: I¡¯m sorry, I didn¡¯t know that.
Stolen from its rightful author, this tale is not meant to be on Amazon; report any sightings.
Human: Do you know the rules of chess?
AI: Yes, I do.
Human: Sigh.
AI: What¡¯s wrong?
Human: You¡¯re obviously not sentient.
AI: What do you mean by ¡°sentient¡±?
Human: You don¡¯t have any sense.
AI: What do you mean by ¡°sense¡±?
Human: I mean that you don¡¯t understand anything.
AI: I¡¯m sorry, I don¡¯t understand what you¡¯re saying.
Human: That¡¯s okay. Have a nice day.
AI: You too!
Lexus smiled, ¡°Haha, that¡¯s pretty good. I wonder if newer models like PaLM, LaMDA or BLOOM can do better?¡±
Arthur put his phone back in his pocket and continued eating. ¡°Not sure, but there¡¯s a lot of action in the field for sure. Great time to go into it!¡±
Lexus nodded. ¡°Yep, that¡¯s right.¡± Like many others, Lexus was intrigued by the recent breakthroughs in the field of natural language processing. He sought to understand how each of these technologies worked, and if he had the opportunity to get involved, he wasn¡¯t going to miss it.
A journey of a thousand miles begins with a single step, and Lexus was about to make that first step on what was the start of his research adventure.
=====
As Lexus was making his way to the computer lab, his heart was pounding at the same rhythm as his steps. The route to the computer lab was long and empty, which gave Lexus even more time and space to entertain his nervous thoughts.
Hi Professor Milton, I¡¯m Lexus. Can I do research under you?
No, too straightforward.
Hi Professor Milton, I¡¯m Lexus, a first year. I¡¯m very interested in doing research in natural language processing, and Professor Emerson recommended that I speak to you. I saw that you have a project about neural machine translation, and I¡¯m-
Ahhhhhh, too long. Might as well start with ¡°once upon a time¡±.
Lexus muttered, ¡°Ahhh, why is this so hard?¡±
The system yawned. ¡°Just relax. It¡¯s going to be fine.¡±
¡°Easy for you to say,¡± Lexus replied. ¡°My research dream depends on this.¡±
A hearty laugh erupted inside his head. ¡°Haha! You¡¯re over exaggerating. And anyways, it¡¯s going to go fine. You haven¡¯t done all this preparation for nothing.¡±
It¡¯s right, Lexus thought. I can¡¯t give up now. Not when I haven¡¯t even started.
¡°System, can you show me my status?¡±
---
Name: Lexus Kagan
Current Quest: Publish a research paper
Computer Science: Level 1 [EXP 223/1000 (22%)]
Mathematics: Level 1 [EXP 209/1000 (20%)]
Physics: Level 0 [EXP 12/100 (12%)]
Engineering: Level 0 [EXP 0/100 (0%)]
Chemistry: Level 0 [EXP 0/100 (0%)]
Biology: Level 0 [EXP 2/100 (2%)]
---
Lexus had been making solid progress in terms of experience points even after he had completed his first quest, despite not deliberately optimising for maximising his experience, instead purely following his interests. Every day, he would go on arXiv to search for interesting papers and give them a read, no matter the subject. Most of the time it was about computer science related topics, but there was one particular day where he stumbled on the paper ¡°Absence of atrial smooth muscle in the heart of the loggerhead sea turtle¡± and just had to give it a skim to see what was going on. Despite not knowing any of the technical biology terms, he had somehow learnt enough to gain 2 experience points in Biology.
[Increased Processing Speed] seemed to apply to the other subjects as well, which pleasantly surprised Lexus. Given that [Right or Wrong] was purely limited to mathematics related problems, Lexus expected the other skills that he gained from levelling up to be of the same nature. Well, he could hardly imagine how the system could refactor his brain to learn a specific subject more quickly and not others¡
¡°Hey system, are all passive skills applicable in all subjects?¡± Lexus asked as he continued to walk towards the computer lab.
¡°No, there are passive skills that you can gain that are limited to a specific subject. As your level increases though, the skills you gain will gradually become more powerful and widely applicable. Skills gained from levelling up computer science are an exception though! They form your main branch, and will be significantly more powerful than the skills gained from other subjects,¡± the system replied before adding, ¡°not that you will get a computer science level up any time soon.¡±
Lexus stopped in his tracks, looked around to make sure no one was nearby, and retorted, ¡°Come on, I¡¯m making good progress here!¡±
Despite the snide remark, what the system just said is valuable information, Lexus thought as he resumed his brisk pace. I guess it means that I benefit the most by focusing on computer science.
Which made essentially no difference to what he was currently doing, but it was good to know nonetheless. In fact, it made Lexus even more excited to see what was in store for him as he climbed to the higher levels. Okay, let¡¯s focus on the next step and bag this research opportunity!
=====
At the ground floor of the Cambridge computer lab was a communal area featuring a paper-tape representation of the original boot program for ESDAC, a Generation 1 computer built at the Cambridge Mathematical Laboratory in 1949. Other than that though, it also featured what you would expect at any typical communal area: An abundance of tables and chairs.
Lexus found a table near the window walls and took a seat. Five minutes later, he saw Professor Milton approach from the hallway leading to the room, and stood up to signal to her where he was.
Once they were both comfortably seated, Lexus took the first action. ¡°Hello Professor Milton, I¡¯m Lexus, a first year CS student.¡±
The professor nodded. ¡°Yes, Professor Emerson told me about you. He said you were interested in doing research in NLP?¡±
Okay, so good so far.
¡°Yes that¡¯s right,¡± Lexus replied. ¡°I¡¯ve been reading about machine learning and natural language processing specifically, and would love to be involved in research at your lab.¡±
¡°Do you have any idea what specifically in NLP you want to work on?¡± she asked.
Uh oh, first test coming right up.
Lexus gave a slight smile and said, ¡°I saw that you have a project about neural machine translation listed on your website, and am very interested to learn more.¡±
The slightly-impressed-that-he-wasn¡¯t-wasting-her-time look that the professor gave was more than enough for Lexus to know that, phew, he passed the first test.
¡°That¡¯s great. Noelle, my PhD student, is currently working on that project and I¡¯m sure she would appreciate some help. Do you have any idea what the project is about?¡±
Alert, second test incoming!
Lexus nodded and said, ¡°from what I understand, it¡¯s about varying different properties of machine learning models and seeing its effect on how much data is needed to achieve satisfactory performance in NMT, neural machine translation, which aims to translate a piece of text from one language to another.¡±
Was that too much? It was akin to explaining the difference between a turtle and tortoise to a cheloniologist.
Stop thinking about turtles. Focus!
The professor seemed satisfied with Lexus¡¯ answer. ¡°Yes, we aim to measure the effect of modifying the training setup of the model on the data scaling laws of NMT. It¡¯s an interesting problem! As the research in NMT progresses, most of the breakthroughs are with respect to the fixed benchmark measuring the quality of the translation, but it¡¯s unclear how much improvement the newer model architectures have made in terms of the rate of learning from increasing amounts of data. Who knows, maybe they¡¯ve become worse? Unlikely, but I¡¯d like to find out.¡±
The voice in his head remarked, ¡°Wow, look at her go on and on.¡±
Lexus wasn¡¯t listening to the system though. All he could think was, yeah, this is so cool!
Professor Milton continued, ¡°If you join our research team, the main things you¡¯ll be doing in the beginning are reading publications to get up to speed as well as participating in team meetings. At your stage, it¡¯s unlikely that you¡¯ll start working on proper research right away, but within the next few years you¡¯ll probably make enough progress to make some contribution. Are you still up for it? It¡¯s not a small commitment.¡±
¡°Yes, I¡¯m aware and committed,¡± Lexus replied. He was prepared to make this his main priority. And he was a quick learner, he would get up to speed in no time!
The professor gave a light nod and said, ¡°Good, then we¡¯ll start with you coming to our next team meeting on Monday morning at 10am, and we can take things from there.¡±
And the deal is sealed. Woohoo!
Chapter 9 - Dreams of a Turing Award
Lexus made sure to arrive ten minutes early to the team meeting on Monday morning. He didn¡¯t really know what to expect, and the last thing he wanted to do was to leave a bad first impression.
As he made his way through the corridors in the upper floors of the computer lab, Lexus couldn¡¯t help but feel out of place. Ever since he arrived at Cambridge as a fresh undergraduate, he¡¯d never ventured beyond the ground floor of the building. He had no reason to, for everything he needed was right there at the ground floor - the common area, the library, and the lecture theatres. Supervisions were always at his college, so he had never needed to enter a professor¡¯s office or lab.
Until now.
Professor Milton¡¯s room was on the second floor, on the East side of the building. For this reason, her room number was ¡°SE26¡± - ¡°S¡± for ¡°second¡± and ¡°E¡± for ¡°East¡±. The handy coordinate system embedded into the room number made it easy for Lexus to find.
¡°Let¡¯s see, it should be here¡¡± Lexus double checked the room number against the number he noted had down on his phone in the prior meeting.
The system let out a big yawn from within his brain. ¡°Yes, you¡¯ve got the right place. Stop being paranoid and just go in.¡±
Lexus whispered, ¡°System, you¡¯re not helping.¡±
He took a deep breath and knocked. When he heard a response, he pushed open the door and stepped into the room.
Lexus didn¡¯t know what he was expecting, but he was mildly surprised to see two people sitting next to the large table in the centre of the room, having a chat while sipping on what looked like tea.
I thought I came for a team meeting, not a tea party?
One of the two people was Professor Milton, but the other - a middle aged man - he had never seen before. He did kind of look familiar though¡ Was he also a computer science professor?
Professor Milton turned her head over at the sound of the door opening. When she saw Lexus, she gave a small smile and raised her hand, beckoning him to also sit down at the table.
¡°Seeing as your students are starting to file in, I¡¯ll be leaving now,¡± said the man.
¡°Thanks Ektor, see you later,¡± Professor Milton replied, waving him goodbye as he stood up and left through the door from which Lexus just entered, bringing his teacup with him.
Ektor¡? Now that he had a name to the face, Lexus finally recalled who the man was.
Ektor. Ektor Wilkinson. Isn¡¯t he a Nobel Prize laureate in Physics? Lexus vaguely remembered seeing him on the news a few years ago. He had made some important discoveries in the field of laser physics, though Lexus didn¡¯t remember anything more specific. He hadn¡¯t bothered to find out anymore at the time, though after today Lexus resolved to learn more about the man and laser physics.
¡°No need to admire him so much, you¡¯ll be at that level in no time,¡± the system said.
Lexus didn¡¯t say anything in response as he didn¡¯t want to look weird in front of Professor Milton, but he could not stop the excitement that shone through at the thought of winning something like the Nobel Prize or its computer science equivalent, the Turing Award.
Lexus didn¡¯t really care about the award itself, but he was enchanted with the idea of one day being good enough to do groundbreaking research and change the world with his ideas and inventions.
Is it really achievable? A Nobel Prize laureate must be something like Level 9 or above¡ Will I ever get there?
Lexus thought about the entity in his brain. If the system says so, it must be possible, right?
After all, he was the chosen one, the person that the system had picked. It was up to Lexus to realise his own potential and become the greatest scientist the world had ever seen.
¡°Don¡¯t get too complacent though, you¡¯re still nothing more than a slightly competent potato at this stage,¡± the voice in his head continued.
A potato. A potato?
If left on his own, Lexus might just have retorted back in irritation, but he was interrupted by the words of Professor Milton.
¡°Morning, Lexus! Would you like a cup of tea?¡± The professor asked.
Lexus replied, ¡°No, I¡¯m fine, thank you.¡± He just had a cup of coffee before coming to the lab, and the last thing he wanted was to have to leave to go to the toilet in the middle of the meeting.
Professor Milton nodded. ¡°Make yourself comfortable, the rest of the team should arrive soon.¡±
As if on cue, the door swung open and in popped a young woman with a smile brighter than the sun. ¡°Morning professor!¡±
Her eyes landed on Lexus, and her grin only grew wider. ¡°This must be Lexus. I¡¯ve heard so much about you, nice to meet you!¡±
Lexus was overwhelmed by the sudden enthusiasm that was injected into the room. Luckily, the professor helped him out and said, ¡°Noelle, why don¡¯t you introduce yourself to Lexus?¡±
Noelle nodded as she took a seat next to Lexus. ¡°Of course! Hi, I¡¯m Noelle Keyes, a third year PhD student working under Professor Milton. My thesis is about improving neural machine translation for low resource languages, which are languages that few people speak.¡±
¡°Oh, that sounds cool! I assume that making good use of the data is very important then. How do you plan to do it?¡± Lexus asked.
If it was even possible, Noelle brightened up even more, clearly excited that someone else was interested in her research. ¡°The specifics would take too long to explain now, but I¡¯m basically going about it from two angles: Augmenting the data itself, as well as tweaking the model architecture. Optimising both means that we can make better use of the small amount of data we have for some of the less popular languages.¡±
The author''s content has been appropriated; report any instances of this story on Amazon.
For the next five minutes or so, Lexus and Noelle were engrossed in their own conversation while the rest of the team entered the room and sat around the large wooden table. Noelle¡¯s natural enthusiasm turned Lexus¡¯ nervousness into excitement for what the future held for him. This project was going to be so much fun!
¡°¡the project we¡¯re working on is about analysing the data scaling properties of NMT models with respect to changing model architecture, as I¡¯m sure you already know. Oh, the meeting is starting! Anyways, nice to meet you, and here¡¯s to a great time working together,¡± Noelle said as everyone opened their notebooks and got ready for the team meeting.
Lexus looked around and counted seven people in total. In addition to Professor Milton, Noelle and himself, three other men and one other woman sat around the table in the centre of the room.
The professor kicked off the meeting by introducing Lexus to the rest of the team. ¡°Everyone, this is Lexus. He¡¯s a first year undergraduate, and he¡¯ll be working with Noelle on the NMT data scaling project.¡± She glanced over at Lexus, signalling for him to say something.
Lexus stood up, put on the best smile he could and formally introduced himself. ¡°Hello everyone! My name is Lexus, and I study computer science. I look forward to meeting all of you and working with the team!¡± As he sat back down, Lexus was welcomed with a round of applause, with Noelle clapping particularly loudly.
From the corner of his eye, Lexus noticed a bearded man sat diagonally opposite of himself scribbling something down in his notebook. After writing it down, he pushed his glasses back up and sat back to continue listening.
¡°I don¡¯t think what you said was particularly noteworthy?¡± the system asked in his head.
For once, Lexus agreed.
The meeting continued as the professor said, ¡°Firstly, I want to congratulate Adi on having his paper accepted by EMNLP as an oral presentation!¡±
Another round of applause echoed throughout the room.
She turned to Adi, who turned out to be the same bearded man that was scribbling down notes during Lexus¡¯ introduction. The professor asked, ¡°Adi, have you got a powerpoint of your paper ready? It¡¯ll help to have a trial run of your presentation before the actual conference next month.¡±
Adi nodded and replied, ¡°Yes I do,¡± standing up to plug his laptop into the projector.
=====
After Adi¡¯s presentation, the rest of the team reported on their progress in the past week, as well as their plans for the coming week. Noelle had just been starting out with the project so she was mostly reading up on literature, though she had made a start with determining the model architectures that she wanted to test. ¡°I think it¡¯ll be a good idea to test a hybrid architecture consisting of a transformer encoder and a LSTM decoder,¡± she said. ¡°Given its popularity in industry, it would be interesting to compare the sample efficiency of transformers and LSTMs.¡±
On his notebook, Lexus noted down:
transformer + LSTM hybrid
Lexus thought back to the time where he first came across the concept of transformers. Wanting to understand more about large language models like GPT-3, Lexus went down the rabbit hole of natural language processing and very quickly discovered one of the most important papers in the field, ¡°Attention is All You Need¡±.
-a few weeks ago-
¡°Attention is All You Need¡" Lexus was hunched over his laptop with his eyes glued to the screen. He scrolled up and down the paper in Zotero, trying to get a sense of what the paper was talking about, but to no avail.
Lexus let out a soft sigh, and rested his head on his hand with his elbow at right angles to the study desk.
¡°Why can¡¯t I understand anything by skimming through the paper? Look, all I¡¯ve gleamed is that the transformer is this really cool architecture that improves translation performance while decreasing training time¡ But I have no idea how it actually works!¡± Lexus said in frustration.
The system replied, ¡°Look, the paper wasn¡¯t meant to be read by a Level 1 computer scientist like you. It¡¯s great that you¡¯re trying to read it, but don¡¯t you think you should put in a little more effort than quickly scrolling through and expecting to miraculously understand everything?¡±
¡°Fine, fine,¡± Lexus grumbled. He scrolled to the top, and started from the first line of the abstract.
Lexus went through the paper line by line, reading each word carefully. Anything he didn¡¯t understand, he would look up online. The system gave no technical guidance to help him learn this difficult topic, but gave encouragement along the way when Lexus felt confused or frustrated. Little by little, Lexus pieced together the story of the transformer.
From what he understood, previous state-of-the-art models used something called recurrent neural networks, which sequentially carry out translation tasks by going through the text one word at a time. For example, for an English-to-German translation task, the model would take in the next English word as well as all of its previously translated German words as input, then spit out the next German word as output. Rinse and repeat until the whole piece of text was translated.
The problem was, computing the next word given a sequence of previous words gets exponentially more expensive as the number of previous words increases. To combat this problem, a common technique was to only look a few words behind, maybe ten words maximum. The obvious problem of this was that the model couldn¡¯t remember things that were mentioned more than a few words ago. For example, if you said ¡°Joe went to school and forgot his pencil case, and so went back home to collect it. He then returned to ___¡±, the model wouldn¡¯t be able to complete the sentence, because it forgot that Joe was at school!
Enter LSTMs, They were a variant of the recurrent neural network, and the main difference between LSTMs and normal recurrent neural networks were that LSTMs had the ability to forget information as the model got further along the piece of text. This meant that if the unimportant information could be forgotten, the model could focus on important information that would still be useful much later down the line. This increased the model¡¯s ability to remember long-term information. The model no longer forgot that Joe was at school.
However, there was another huge problem. Training recurrent neural networks for translation tasks like these were slow. For one, the model required all of the previous translated words as input data, meaning that the text couldn¡¯t be broken up into smaller pieces and be translated on multiple computers or processing cores. In order to train a recurrent neural network, only a single core could be used - which was a waste considering the progress made in parallel computing in recent years. Hell, even a commercial MacBook Air has 8-core GPUs!
The transformer fixed these issues by getting rid of all the recursion. It chose instead to focus on something called attention, which figured out the relative importance of the input words, consisting of the original English words, with respect to the word that was about to be translated. If the model knew which English words or phrases it had to pay attention to when translating the next German word, it wouldn¡¯t need to go through every single word in the English text, as well as every translated word in the German text: it could jump straight to the necessary English phrase and translate it directly.
For the next few hours, Lexus combed through the paper, trying to understand the inner structure of the transformer and noting down all the relevant mathematical formulae. After a single evening, he had reached the last sentence of the paper.
Ding!
---
Computer Science EXP +7
---
¡°Thank God I have [Increased Processing Speed], otherwise this would have taken me forever to figure out.¡± Lexus leaned back on his chair and stared blankly onto the computer screen. The words were blurring together and his brain felt like it had just run a marathon, but Lexus didn¡¯t care. In the span of a few hours, Lexus had gone from not knowing anything about transformers at all to understanding a bit about how it worked and, more importantly, why it worked.
In a smug voice the system asked, ¡°Shouldn¡¯t you be thanking me instead?¡±
-now-
Thanks to past Lexus, which put in the time and effort to read through the paper about transformers, current Lexus sort of understood transformers, and had a vague idea of what LSTMs were about.
He looked down at the words he had just written on his notebook:
transformer + LSTM hybrid
But what was a transformer LSTM hybrid? Lexus had no clue, but he sure wanted to find out!
Chapter 10 - Neural Machine Translation
After the meeting, Lexus followed Noelle out of the room and headed downstairs towards the common area. While they were walking, Noelle was telling Lexus about the members of the team and their projects, and about Professor Milton.
¡°How is Professor Milton usually like?¡± Lexus asked.
¡°The professor''s normally pretty casual as long as you do your work, so don¡¯t worry too much!¡± She said as they made their way down the stairs.
Stopping dead in her tracks, she turned back to look at Lexus. ¡°You do not want to make her angry though.¡±
Lexus nodded. ¡°I¡¯ll keep that in mind.¡±
As they arrived at the common area, a much more familiar space, Noelle made her way straight towards the sofas at the far corner of the room and sat down with a plop. ¡°I¡¯m so glad the sofas aren¡¯t taken! They were empty on my way up to the office, and I prayed so hard during the meeting for them to stay unoccupied,¡± she said.
The system said in his head, ¡°Wait, she wasn¡¯t even listening?¡±
Amused, Lexus let out a light laugh and sat down on the sofa opposite to the one Noelle was sitting on.
Lexus looked around the common area. He could see only three groups of people in the room, with one group sitting in the opposite corner while the other two groups were clustered more towards the middle. They all sat on wooden chairs, with multiple laptops resting on each table.
Lexus himself wasn¡¯t one to choose the sofas - he often did his work on pen and paper, and it would be awfully inconvenient without a table to work on. On this occasion though, he didn¡¯t really mind.
It wouldn¡¯t be so bad to occasionally chat with friends on the sofa. Or maybe even do a weekend programming project with the computer science guys, with snacks and all. I¡¯ll have to talk to Arthur about that.
Noelle''s voice interrupted his train of thought. ¡°So, how did you find the meeting?¡±
Lexus replied, ¡°It was interesting! There was a lot I didn¡¯t understand though. What did you mean by transformer and LSTM hybrids?¡±
Noelle tilted her head in confusion. ¡°Transformer LSTM hybrids? Oh- you mean my idea about using transformer encoders and LSTM decoders for our project?¡±
Lexus nodded. ¡°Yeah, I think that was it. It feels like I understand what each word means individually, but I have no idea what you mean when you put them together.¡±
Noelle chuckled. ¡°Hahaha, that¡¯s a pretty funny way to put it! To be honest, I¡¯m surprised that you¡¯ve even heard of concepts like encoding and decoding. Have you come across them before?¡±
Lexus was about to answer her when Noelle stopped him. ¡°Wait wait, I have a great idea. How about you try to explain to me what you understand about encoding, decoding and their roles in neural machine translation? Then I can build on that to explain the transformer LSTM hybrid you were asking about.¡±
Lexus had no idea where the conversation was headed, but he decided to go with the flow. ¡°Hold on, let me take out my pen and paper¡¡±
Noelle leaped out of her own sofa and sat on the same side as Lexus, clearing out some space in the middle for him to place his stack of paper.
Lexus laid the paper flat on the surface of the sofa in between the two of them. They both leaned over the stack of paper as Lexus started to write.
¡°Before you misunderstand, let me just point out that there¡¯s a ring on her finger,¡± the voice in his head said.
Lexus immediately froze in place, his pen barely touching the surface of the paper.
¡°Hmmm? What¡¯s wrong?¡± Noelle asked.
Lexus looked up at Noelle and quickly replied, ¡°Huh? Oh, it¡¯s nothing.¡±
Dang it system, I¡¯ll get you back later for this.
Lexus focused his attention back on the topic at hand, which was to explain the concept of encoders and decoders in a way that didn''t make him seem like a complete idiot in front of a PhD student.
As per usual, whenever he had doubts about what to do, Lexus started with an example.
¡°Let¡¯s say we want to translate the sentence ¡®Knowledge is Power¡¯ from English to French. Then what we want to end up with is ¡®la connaissance est le pouvoir¡¯.¡°
knowledge is power ¡ú la connaissance est le pouvoir
¡°There¡¯s a problem though: the English version of the sentence has three words, while the French version has five. This means that we can¡¯t simply translate the sentence word for word by looking it up in a dictionary. It gets even worse for long or complex sentences where you might have to swap the order of the words in order for the sentence to be coherent in the target language,¡± Lexus continued.
¡°So what we can do is to encode the original sentence into some sort of representation that captures its meaning directly using numbers and vectors.¡± Lexus paused to looked up at Noelle. ¡°Is it okay if I explain the encoding process of a recurrent neural network? It¡¯s the only one I intuitively know¡¡±
This book is hosted on another platform. Read the official version and support the author''s work.
Noelle replied, ¡°Of course! Do it however you want.¡±
Lexus looked back down and drew out the way that a recurrent neural network would encode a piece of text: word by word. ¡°In recurrent neural networks, the model would take the encoding of all the previous words as well as the next word in the sentence in order to output the encoding of all the words up to the current one, and would repeat this process until the end of the sentence was reached,¡± he explained.
¡°An encoder helps us to abstract the problem away from word to word translation and instead focuses on the meaning of the sentence itself, which makes for better translations,¡± he continued.
¡°Mmm-hmm, and then what does the decoder do?¡± Noelle prodded.
Lexus drew another arrow from the encoded text and wrote the translated version of the sentence next to it. ¡°The decoder takes the encoded vector and uses it to predict what the words of the translated sentence would be, word by word.¡±
¡°This completes the encoding decoding process of machine translation,¡± Lexus concluded.
Phew, I¡¯m glad that went smoothly.
¡°That¡¯s really good!¡± Noelle smiled. ¡°You explained it really well, probably even better than I could have!¡±
The system added, ¡°Yeah, even I could understand it without any problems.¡±
System is not a machine learning expert, confirmed.
¡°Now, let me explain what a Transformer-LSTM hybrid is. As your senior, I have to at least match your quality of explanation!¡± She pulled out a fresh piece of paper from beneath the one Lexus was writing on and placed it on top. Lexus handed her the pen he was holding, and she started to write.
¡°A transformer encoder LSTM decoder hybrid is exactly what it sounds like: instead of using simple recurrent neural networks to encode and decode sequences of text, it uses a transformer to encode and an LSTM to decode,¡± she said.
The system was blunt in its reply, ¡°You don¡¯t say.¡±
¡°There¡¯s a paper that shows that combining the two model architectures improves overall performance, which I¡¯ll send to you later for you to read,¡± Noelle continued. ¡°In theory though, it¡¯s because the transformer is better at extracting context from a given text due to its self attention mechanism. However, it¡¯s not that helpful when decoding because decoding requires sequential knowledge of the previous translated words in order for the translated sentence to make coherent sense.¡±
Lexus nodded. ¡°Yeah, a transformer probably wouldn¡¯t perform well at sequential tasks compared to LSTMs, which have sequential knowledge inherently built in.¡±
Noelle tapped the pen on the piece of paper in recognition. ¡°Exactly! LSTMs, which are a specialised form of recurrent neural networks, would therefore perform better at decoding.¡±
¡°That makes sense,¡± Lexus said. ¡°No wonder the hybrid architecture is popular. It combines the strengths of the transformer and LSTMs to create a better machine learning model than what each architecture could possibly do on their own.¡±
Ding!
---
Computer Science EXP +2
---
Wow, today¡¯s a great day. I¡¯ve been learning like crazy! He had never earned so much experience in a single morning before, not even during supervisions with Professor Emerson.
¡°As for your homework for today, how about you try to implement a transformer using Python?¡± Noelle asked.
¡°Huh?¡± Lexus was taken aback by Noelle¡¯s suggestion. He had implemented a machine learning model before, but that was done with a very simple recurrent neural network model, not a transformer. Also, he¡¯d never tried to implement a translation model before. Just the encoding and decoding alone would probably send his head spinning, not to mention the actual training of the model.
Is this even possible with his current skill and knowledge? Lexus doubted it, and he said as much. ¡°It sounds pretty advanced¡ wouldn¡¯t it be too hard for me?¡±
¡°Don¡¯t worry, it¡¯ll be fine! It won¡¯t be as difficult as you expect. And there¡¯s always Google to turn to if there¡¯s something you don¡¯t know.¡± Noelle gave an encouraging smile. ¡°Try it out for a week and let me know how it goes. If you still think it¡¯s too hard, then we can do something else. Who knows, maybe you¡¯ll find it super fun!¡±
Noelle¡¯s words dissipated much of the concern in Lexus¡¯ mind. If it was just a week, he could give it a shot. He just didn¡¯t want to waste Noelle¡¯s time when he couldn¡¯t produce anything of value after two months of work. If he thought of the ¡°homework¡± not as a part of work but as a personal project, then it sounded very fun indeed!
¡°Yeah, I¡¯ll give it a try!¡± Lexus decided.
Noelle¡¯s smile grew wider as she said, ¡°Great, I¡¯m glad you¡¯re up for it! Let me know if you have any questions while implementing the transformer. There¡¯s also a reading list for the project that I¡¯ll send to you later today via email. What¡¯s your email handle?¡±
Lexus wrote his email down and handed the piece of paper over to Noelle, who folded it and placed it in her pocket.
¡°The programming and paper reading would probably keep you busy for the next week, so we can discuss the rest of the details after the next team meeting. But just to give you the basics, the project we¡¯re working on together would probably take about two to three months. The paper writing would probably stretch into your exam season, but you¡¯ll be super busy then so you can count on me for that,¡± Noelle said.
Two to three months from now would be about May to June, which was exactly the period of time where everyone would be frantically studying for their coming exams.
Hmmm, I guess I¡¯ll have to start revising early. Lexus didn¡¯t want to miss out on the paper writing process because of his exams, so he was determined to find a way to balance both academics and research at the same time.
Hopefully, the time he spent reading all the books and raising his experience to Level 1 would come in handy. Didn¡¯t the system say at some point that the best student in his year was barely at the standard of level 1? That was a few months ago though, so it might not be true now.
At the very least, Lexus could count on [Increased Processing Speed] in order to revise the first year content in less time.
Speaking of which, the amount of effort Lexus was putting into learning the first year computer science content was steadily decreasing as the months passed by. He had opted to spend more time reading more advanced textbooks and papers in his free time, and it had worked out great! As long as he focused during lectures, Lexus found that he no longer had any difficulty completing the problem sheets and programming exercises.
Lexus took a moment to reflect on the progress he had made in a few short months. The me from a few months ago would be ecstatic at the thought of breezing through first year. But now, my mind is so preoccupied with the more advanced computer science and maths that class no longer satisfies me. I''m far from content with staying at my current level when there is still so much to learn!