1. Metasploit polymorphic encoder
The metasploit framework comes with an excellent set of tools that includes a polymorphic encoder that can "encode" your trojan or virus such that antiviruses would not be able to recognize its signature, thus avoiding detection. You need Backtrack or Kali to do this. The best encoder under msfencode is shikata ga nai, which can be used to "encode" and obfuscate your payload(read trojan) multiple times.
A detailed tutorial on this can be found here. Also, here is a video tutorial on metasploit backdoors and encoders.
Crypters and packers are specialized tools that can encrypt and pack your payload(read trojan or virus) so that antivirus cannot get to the actual core of the payload. The encryption on the payload prevents antiviruses from peeking inside. Because of this, antiviruses will not detect your trojan. Once you are ready to deploy the trojan, the payload gets unencrypted and unpacked to unleash all malicious activities and infections.
There are many crypters available online both free and paid. However, a note of caution is that crypters are not so reliable. There are chances that the crypters will not work. You can even create your own crypters if you are good at programming.
Antiviruses use file signatures to detect viruses. These file signatures are unique patterns inside the virus. They are very small(a few dozen bytes). Binary editing involves finding the file signatures and directly altering it. Once the contents of the file signature are changed, the antivirus will no longer recognize the signature.This can be done using a Hex Editor.
For more info, refer to the paper Taking Back Netcat
4. Modify Source code
If you have the source code of the program/virus, you can modify it. For instance, If there is a switch case condition in the code, convert it into if-else. This should not affect the functioning of the program in any significant way. There are many other changes that you can make, like changing variable names, upper to lower case etc.
can be changed to
5. Recompile the payload/Use an alternate version
If you have the source code of the virus, you can recompile it using a different compiler. It will create a completely fresh executable. That way, there is a chance that the antivirus will no longer recognize the signature.
If you are using a readymade program, then try using a different version. For example, if you are planning to install netcat on your victim's system and the antivirus is flagging it, then you can use other editions of netcat (nc.exe) like mocat, netcat2, cryptcat etc.
Keeping in mind that these techniques are not sure to work every time, if you still didn't succeed, you can always write your own virus/trojan/keylogger from scratch. That is the only way of avoiding antivirus signature based detection. Again, you must be careful in writing the code because antiviruses also have behavior-based detection also known as heuristics detection.